The Typical Data Problems in Retail

In almost every first conversation, someone eventually says: our situation grew organically over time — it's a special case. That's half true. Organic growth happens everywhere. A genuinely special case is rare. Anyone who has seen many retail system landscapes from the inside recognizes the same six patterns, varying in intensity but appearing with reliable regularity.

That familiarity is actually good news. Known patterns have known causes and known remedies. A walkthrough of the six most common patterns is worthwhile for two reasons: you'll recognize your own organization in them, and you'll know what to look for in your own systems.

Why do data problems in retail look so similar?

These system landscapes share the same history. Almost every retail business has followed the same path: it started with an inventory management system — often an ERP — that handled everything. Then came an online shop requiring richer product data, so a PIM was added. Later came marketplaces, an app, a new point-of-sale system, a warehouse management system. Each new system solved a real problem and brought its own data storage with it.

The result is consistently the same: the same information lives in multiple systems, and the connections between them have grown over the years rather than being planned. In a Gartner survey, 63 percent of data management leaders said they either lack appropriate data management practices for AI or aren't sure whether they have them. That finding is no surprise to anyone who knows the shared history. It's the logical result of how these landscapes came to be.

What does duplicate maintenance in the product master cause?

It creates two versions of the truth about the same product — and nobody can say which one is right. This is pattern one, and it has the most far-reaching consequences. A typical setup: the ERP manages the product for purchasing and replenishment, while the PIM manages it for the shop. Both systems are maintained by different teams, at different times, according to different rules. Without a single authoritative system, the records drift apart.

The effects are measurable: duplicate records — created when a team can't find a product and sets it up again. Conflicting attributes, when both systems hold different values for the same field. And a gradual erosion of trust, because everyone knows the data can't be fully relied on, even if nobody can pinpoint where it breaks down.

What's telling is how this pattern develops. It almost always starts with an interface that was supposed to keep things in sync automatically. Then a case came along that the interface couldn't handle, someone updated it manually, and the exception became standard practice. Years later, nobody knows which fields the interface controls and which ones are maintained by hand. A quick test for your organization: ask purchasing and e-commerce separately which system is the master for the product record. If you get two different answers, you have pattern one.

How do you spot dead required fields?

By the difference between filled in and usable. Pattern two is more deceptive than empty fields, because at first glance everything looks fine. A required field forces an entry when a product is created — so something gets entered: 999 as a lead time, a period as a description, the creation date as a placeholder. The field is filled, the completeness check is satisfied, and the value is still useless.

System migrations are a particularly rich source of dead fields. When moving to a new system, legacy data had to meet the new required field rules — so missing values were filled with defaults, with every intention of correcting them later. Later never came. Anyone who wants to know how old this legacy is can sort the suspicious values by creation date.

Two straightforward checks can uncover this. First, the fill rate per field, which shows where nothing has been entered. Second, a plausibility check, which shows where nonsense has been entered: values that appear suspiciously often, values outside reasonable ranges, identical values across thousands of products. Both checks run directly on the systems and take hours, not weeks. They belong among the first steps of any serious data review.

How do silos between store and online develop?

Through separate systems for the same people and the same products. Pattern three has two faces. On the customer side: the store knows its customers from receipts and perhaps a loyalty card; the shop knows them from accounts and orders. The same person exists twice, with no connection between the two records. Any cross-channel analysis, any personalization, and any honest view of revenue per customer breaks down at this divide.

On the product side: store inventory and online inventory are managed separately and updated at different frequencies. The shop promises availability that the store no longer has, or conversely, stock sits in the store while online it shows as sold out. Anyone offering click-and-collect or ship-from-store feels this silo every day in cancellations and disappointed customers.

Behind both faces lies the same organizational pattern: store and online have grown as separate divisions, with their own budgets, their own systems, and their own metrics. The data separation simply mirrors the organizational separation — which is why it can rarely be resolved through technical means alone. A customer matching project doesn't fail because of the matching algorithm; it fails because of the question of who owns the customer record afterward.

What other patterns are there?

Three more that get less attention but cause just as much damage. Pattern four is knowledge locked in spreadsheets: supplier terms, assortment rules, seasonal logic — maintained in files on shared drives, in multiple versions, with nobody knowing which one is current. Workable for humans, invisible to automated systems.

Pattern five is the absence of ownership. No data domain has a named person responsible for it. Corrections happen ad hoc, from multiple departments in parallel, and any cleanup degrades again within months because nobody is defending the maintained state. In our assessments, this is the most common root cause behind all the other patterns.

Pattern six is latency — the delay between an event and the data reflecting it. Last night's store inventory, yesterday's sales, a pricing change that won't appear in the system until next week. Irrelevant for monthly reports, but a real problem for fast-moving assortments and automated decisions. Gartner notes that a lack of AI-ready data fundamentally puts projects at risk — and timeliness is part of what makes data fit for purpose.

Why do these problems stay hidden for so long?

Because people compensate for them every day, without realizing it. The planner knows the lead times are unreliable and adjusts mentally. The e-commerce manager knows which inventory figure to question. Accounting has its own spreadsheet bridge between two systems. Each of these workarounds functions, quietly costs time, and hides the problem from every metric.

This compensation carries a second, often overlooked cost: it depends on specific people. The tacit knowledge of which value is right and which isn't isn't written down anywhere. New employees need months to build it up, and when an experienced person leaves, their correction knowledge goes with them. What looks like a data problem is, in this sense, also a personnel risk.

The damage only becomes visible when automation enters the picture. An automated system doesn't come equipped with the quiet expertise of experienced staff — it takes every value at face value. What was a familiar nuisance becomes a measurable error, repeated every night and across every product. Why this is regularly misread as AI failure — even though the errors were already there before — is something we explored in the article on AI agents in production.

Where do you start if you recognize yourself in this?

With the use case, not a full overhaul. The instinct after a walkthrough like this is to launch a sweeping data quality program covering all six patterns at once. We'd advise against it, and we speak from experience: such programs take years, lose their sponsor along the way, and often leave little that lasts — because without pattern five, clear ownership, any maintained state will deteriorate again.

The more effective approach is the reverse. Choose the initiative with the greatest impact on your business and bring only the data domains that initiative requires into order. Which domains those are, and how to assess their current state yourself, is covered in the article What data does AI need?. Those who want a reliable answer can have the relevant domains measured — with documented evidence from the systems, following the open standard described on the methodology page. The six patterns lose their menace once they've been measured. At that point they're no longer vague legacy problems. They're numbers with a priority order.

Sources

Source	What it says
Gartner · Lack of AI-ready data puts AI projects at risk, Februar 2025
prodct · Beispiel-Report ACME Inc. (fiktive Daten)