Why AI Agents Fail in Production

The story almost always starts the same way. An agent goes into a test environment, the early results look good, and management is satisfied. Six weeks later, a team member is manually reviewing every single decision. After three months, they're back in full control, and the agent is running in name only. At some point, someone quietly switches it off — no project closure, no announcement.

This pattern deserves a name, because it is currently costing retailers a great deal of money. This article examines how common it is, what actually causes it, and how to identify the risk in your own project before the first invoice is paid.

How often are AI agents shut down?

More often than the success stories suggest. Gartner forecasts that more than 40 percent of agentic AI projects will be cancelled by the end of 2027. The firm cites rising costs, unclear business value, and inadequate risk controls as the main reasons. This is a prediction, not a measured result, and should be read as such. It does, however, align with survey data on the current state of affairs: according to S&P Global Market Intelligence, the share of companies that abandon the majority of their AI initiatives before reaching production has more than doubled, to 42 percent.

What stands out about these figures is not their size but their direction. Models improve year on year, yet cancellation rates continue to rise. If the technology were the problem, the curve should be moving the other way. It isn't. The bottleneck lies elsewhere.

What actually causes agents to fail?

The foundation beneath the agent — almost never the model itself. Research by the RAND Corporation surveyed experienced data scientists and engineers about failed AI projects. The most common causes: poorly understood problem definitions and missing or inadequate data. Technical limitations of the models appear far down the list. Gartner adds to the data picture: the absence of AI-ready data fundamentally threatens projects, and a significant share of organisations simply do not have the necessary data foundations in place.

What does this mean in practice for a retail agent? An agent that triggers purchase orders, adjusts prices, or responds to customer enquiries bases its decisions on whatever data the surrounding systems supply — product master data, stock levels, pricing conditions, customer history. If that data is incomplete or contradictory, the agent's decisions will be incomplete and contradictory. Only automated, at scale, and with complete confidence. AI does not solve data problems. It scales them.

Costs belong in this picture too. An agent that executes multi-step tasks autonomously incurs ongoing processing costs — and those costs rise with every correction loop. An agent working from poor data runs many such loops: it makes a decision, a person intervenes, the case goes back, the agent decides again. The tool that was supposed to save work becomes a system that generates work while running up compute costs. The escalating costs that Gartner cites as a reason for cancellation are often just the symptom of these loops.

A second, rarely named cause also plays a role: the absence of controlled operations. Many agents go into testing without a log of their decisions, without defined boundaries, and without a structured exception-approval process. A person who is expected to intervene needs traceable decisions and a clear point of intervention. Without both, all they can do is distrust everything. They end up reviewing every decision, and the agent stops saving a single minute.

Why does this only become apparent in production, not in testing?

Because the test is designed to impress. That is not bad intent — it is simply the nature of a pilot. Someone selects the data, cleans it by hand, and runs the system under supervision. Inconsistencies in product master data, empty mandatory fields, outdated stock levels: all tidied up in advance so the demo runs smoothly. The pilot demonstrates the capability of the model. It says nothing about the state of the organisation's own data landscape.

In production, the manual work disappears — and something else becomes visible. People have been quietly compensating for poor data for years. A buyer, the person responsible for replenishment decisions, can spot an implausible lead time at a glance and mentally correct it. That silent knowledge exists nowhere in any system. The agent takes over the process without it, and exposes hidden gaps for the first time. To everyone involved, it looks as though the agent has introduced new errors. In reality, the errors were always there. No one had ever counted them.

A worked example using fictional data. A multichannel retailer tests a replenishment agent. During a six-week pilot, a buyer intervenes on 31 percent of the agent's order proposals. Analysis of the logs shows that 72 percent of those interventions trace back to master data — primarily missing lead times and minimum order quantities. The agent was never the problem. The full findings are available in the free sample report.

How do you spot the risk before you start?

Look for five warning signs that can be checked without a project or a consultant. Each one has already stopped initiatives in our data reviews.

1. The process the agent is meant to take over has no measured error rate or intervention rate. If you don't know how often corrections are made today, you have nothing to measure the agent against later.
2. The data the agent will use for its decisions is maintained in parallel across multiple systems, with no single system of record. Contradictions then flow unfiltered into its decisions.
3. No one can say how completely the decision-relevant fields are populated. That figure can be queried in an hour. If it's missing, it simply means no one has wanted to know.
4. The knowledge the process depends on lives in spreadsheets and people's heads. An agent cannot read from these reliably, and no one notices when it acts on outdated information.
5. There is no plan for logging, boundaries, and exception approvals. An agent without controlled operations loses the team's trust after the first visible mistake. After that, regaining it is nearly impossible.

If two or more of these apply, the project is not lost. It just has the wrong sequence. Close the gap first, then start the agent. The other way round, you are funding an expensive experiment with a foreseeable outcome.

What do organisations do differently when their agents succeed?

They build reliability before autonomy. That sounds straightforward and is, in practice, the rare exception. The typical pattern is the opposite: high autonomy ambitions, weak foundations. In our reliability-autonomy framework, that sits in the bottom-right risk zone — which is exactly where the projects that end up in the cancellation statistics come from.

Organisations whose agents hold up in day-to-day use follow a different sequence. They first identify which data areas their specific use case requires, and bring only those into shape. Not a company-wide data programme — just the smallest foundation that is sufficient. They launch the agent with tight boundaries and a clear exception-approval process, measure the intervention rate from week one, and expand autonomy only when the numbers justify it. And they treat operations as part of the project: logs are reviewed, and exceptions feed back into data maintenance.

One more point that comes up noticeably often in successful projects: a pre-defined target metric. Before the agent goes live, everyone agrees on which KPI it will be measured against — for example, the out-of-stock rate or time spent on replenishment decisions — and where that metric stands today. After a fixed period in production, typically 90 days, it is measured again. This keeps everyone accountable. Without a target metric, every agent project ends in the same debate: some people think it's working, others think it isn't, and nobody can prove either way.

The difference, then, lies less in the tool than in the sequence. How to determine the right sequence for a specific case is illustrated in our article on agentic replenishment — the use case that most commonly serves as a starting point in retail.

How do you reduce risk before you invest?

With a measurement, not an opinion. Whether your data can support an agent is something you can determine in advance: with evidence from your own systems, following an open standard, in three weeks. The output is a reliability score for your use case, a clearly identified bottleneck, and a prioritised action plan. If the result is good, you move forward with confidence. If it isn't, you know before you invest — not after six months of piloting.

The criteria and formulas behind the assessment are publicly documented and available on the methodology page. What matters is that the outcome is open: an honest review may well conclude that your preferred use case is not worth the investment right now. That answer after three weeks is uncomfortable, but inexpensive. A switched-off agent is the most costly way to find out about the state of your own data. There is a better way to find out sooner.

Sources

Source	What it says
Gartner · Prognose zu agentischen KI-Projekten, Juni 2025
Gartner · Lack of AI-ready data puts AI projects at risk, Februar 2025
RAND Corporation · The Root Causes of Failure for AI Projects, 2024
S&P Global Market Intelligence via CIO Dive · AI project failure rates on the rise, März 2025
prodct · Beispiel-Report ACME Inc. (fiktive Daten)