Data Quality Framework for Machine Learning and AI Agents

Global AI spending hit $684 billion in 2025, and RAND Corporation's analysis found more than $547 billion of it, over 80%, failed to deliver what it promised. MIT's Project NANDA went further in July 2025: 95% of organizations running generative AI saw zero measurable effect on profit and loss. Not a small return. Zero. McKinsey found 88% of organizations use AI somewhere in the business, yet only 39% can point to any bottom-line impact, and IBM's Institute for Business Value put a finer point on it: only 16% of AI initiatives have scaled past a pilot. The pattern is too consistent across industries and model types to be a string of bad executions. Something upstream is broken, and it isn't the models.

How data gets blamed but not actually fixed

Ask a CDO what killed the last AI project and the answer comes fast: data. Informatica's 2025 CDO Insights survey found 43% of respondents naming data quality and readiness as a top obstacle, tied with a lack of technical maturity. RAND's 2024 study, built on interviews with 65 data scientists and engineers, identified inadequate training data, both quality and access to it, as a key root cause of AI failure. Gartner has gone as far as predicting that 60% of AI projects lacking AI-ready data will be scrapped entirely by 2026.

Say those words out loud though, "data problems," and watch how fast they turn vague. In practice they mean something specific and unglamorous. Customer records live in four systems that don't talk to each other. Marketing calls something a "customer" that sales calls a "lead." Ten years of transaction history has three different date formats depending on which system was in use that quarter. The demo dataset that impressed everyone in the sales pitch bears almost no resemblance to what actually sits in the warehouse.

The programs that work tend to spend 50 to 70% of their timeline and budget on data readiness, not modeling: extraction, normalization, governance metadata, quality dashboards, retention rules. That ratio tells you something. Most companies still treat data quality as a box to check before a project starts, a one-time cleanup. But once agents enter the picture, that assumption stops holding. Data quality is now a continuous, ongoing requirement. It's a condition that has to hold every second the system runs.

What data quality has traditionally measured and why those dimensions were designed for humans

IBM defines AI data quality as the degree to which data is accurate, complete, reliable, and fit for use across training, validation, and deployment. Fair enough. But the dimensions that have dominated the field for decades, accuracy, completeness, consistency, timeliness, and related measures, were built for a different job.

Evidence from healthcare, finance, and government all points the same direction: get any one of these wrong and you pay for it in lost money, wasted operational hours, or eroded trust. ISO/IEC 5259-5:2025, published in February of that year, formalized a governance framework for data quality in analytics and machine learning, covering oversight, accountability, and controls across the data lifecycle. Even the standards bodies now treat AI as its own category of quality problem, separate from plain old business reporting.

Look closely at what these five dimensions assumed, though. A human analyst gets handed a spreadsheet, notices a weird column, and walks over to ask someone what it means. Gaps in context get filled by whatever that analyst already knows about the business. A number that looks two weeks stale triggers a gut check before anyone acts on it. And access gets granted to a named person whose intent is, at least roughly, visible to whoever granted it.

None of that holds when the one reading the data is a piece of software. The mismatch here reflects a fundamental flaw that runs deeper than surface polish. It's structural.

How agents consume data differently from human analysts

An AI agent perceives its environment, decides, and acts toward a goal, largely without a person standing over its shoulder. It can't pause mid-task and ask what a poorly labeled column means. It just proceeds, using whatever's in front of it.

Speed changes everything here. A question a human analyst might run once, carefully, an agent might run a thousand times an hour. Any error in the underlying data doesn't just repeat, it compounds, because nothing in the loop is slowing down to sanity-check the output. A stale number and a correct one look identical to an agent. There's no institutional memory whispering that something feels off, because agents don't have a "feels off."

Multi-agent setups make this worse. In an A2A workflow, an orchestrating agent hands off pieces of a task to specialized sub-agents, and each one resolves its own data independently. If two of those agents define "active customer" differently, that inconsistency doesn't stay contained to one output. It rides along through the entire chain.

Then there's the part that should worry people more than it does: agents don't just read data, they write it. They place orders, update records, trigger downstream actions. A read error produces a wrong answer. A write error produces a wrong action that's already been executed by the time anyone notices. Machine learning teams already spend somewhere between 60 and 80% of their working hours on repetitive operational tasks such as monitoring and retraining. Agentic systems are supposed to run without a person there to catch mistakes in real time, which means they need to absorb that same burden with no hands on the wheel.

The upshot: the bar for agent-ready data has been raised to a fundamentally different standard. It's measured against a different set of questions entirely.

Completeness for agents means something harder than "no missing fields"

Ask a traditional data team what "complete" means and they'll check whether the required fields are populated. Done. For an agent, that's the easy part, and it's nowhere near enough.

Completeness for an agent also has to account for a few other things:

• Whether the dataset actually reflects the population the agent is reasoning about, not just whether the rows exist
• Whether label coverage matches what the agent needs to infer, not just what it was trained on
• Whether the relationships between entities are intact, since a schema can be fully populated field by field and still be missing the one join that gives a number any meaning at all

Research out of the Chinese Academy of Sciences (the SciHorizon-DataEVA project) put this on display in scientific AI. A dataset can satisfy every traditional completeness rule and still be useless for a given model, because the feature structure doesn't match what the model needs. A molecular dataset that only stores SMILES strings, for instance, is complete by the old definition but worthless to a model that needs 3D molecular shape. The same research flagged clinical datasets missing socioeconomic variables: models trained on them start attributing patient outcomes to which hospital someone visited, when the real driver was something clinical or social that never made it into the columns. The prediction looks statistically fine. It's scientifically wrong.

So completeness checks for agent-consumed data need to be built around the agent's actual task and reasoning path, not just the shape of the schema. And there's a scale problem sitting underneath all of this: less than 1% of unstructured data sits in a format an AI system can use directly, and this includes PDFs, emails, scanned documents, and chat logs. Schema-level completeness can look perfect while the real incompleteness problem, the one buried in file formats nobody converted, goes untouched.

Freshness requirements shift when an agent is making a decision rather than producing a report

Timeliness, in the old world, meant the data was current enough for whatever report or dashboard it fed. A daily refresh, sometimes even weekly, was fine, because a human was going to glance at the number and factor in that it might be a day old.

An agent setting prices, routing shipments, flagging fraud, or adjusting inventory doesn't have that luxury. A snapshot that's a few hours stale isn't a minor rounding error there, it's a decision made on the wrong facts. Freshness has to be set by what the agent is actually doing, not inherited from a pipeline someone built years ago for a human dashboard.

The architecture that tends to work here relies on continuous replication through Change Data Capture, feeding a stream processing layer that validates and transforms data as it moves, in real time, with the governed result routed to wherever the AI needs it, feature stores, vector databases, or straight to an agent through something like Model Context Protocol. Static pipelines assume the source systems stay put and the schema never changes. Neither is true for long. Schemas shift, sources drop offline, and the agent just sees whatever the last pipeline run happened to capture, stale or not.

Data access needs to happen at the moment of the query, against a live source, not against a snapshot from last night's batch job. What's "fine" for a dashboard is already too old for an agent making a call in the moment. And that's part of why formal agreements between the teams producing data and the systems consuming it have become an increasingly common way production teams try to enforce freshness at the infrastructure level.

Why semantic context is the quality dimension that traditional frameworks never had to name

Every one of the traditional dimensions assumed a human was on the other end of the pipe, someone who already knew what "revenue" meant at this particular company, which customer table was the real one, and how a metric got redefined last quarter after finance changed its mind.

An agent carries none of that. It reads whatever the schema says, and if the schema says nothing useful, it guesses, producing SQL that looks entirely plausible and is entirely wrong. That gap, the one between a raw field and what it actually means to the business, never had a name in the old frameworks because a human always closed it without anyone noticing.

A semantic layer built for agents is what closes that gap on purpose. It's a governed translation layer that turns raw fields and schemas into business concepts an agent can act on: metrics, dimensions, joins, and access rules, all defined once, so the agent picks from a known, approved set instead of writing its own query logic against raw tables and hoping for the best.

The industry consensus is moving toward treating the semantic layer as essential enterprise-AI infrastructure. The industry is already moving that direction: the Open Semantic Interchange initiative launched in September 2025, backed by Snowflake, Salesforce, dbt Labs, Atlan, and others, so that semantic definitions can move between a BI tool, a warehouse, and an AI agent without getting rewritten each time. dbt open-sourced MetricFlow. Snowflake shipped Semantic Views. Databricks built Metric Views. Salesforce launched Tableau Semantics. That's multiple major companies converging on the same missing piece within a short window, which is usually a sign the gap was real.

A Harvard Business Review study from October 2025, sponsored by Cloudera, found 65% of data decision-makers expect agentic AI to augment or replace many business processes within two years. Yet 73% of that same group said preparing data for AI has been genuinely hard. That gap points to how hard preparing data for AI has proved in practice. And it gets worse in multi-agent chains: under Google's A2A protocol, launched in April 2025, agents in a workflow hand off tasks to one another, making consistent metric resolution across the chain a critical requirement. Without shared semantic grounding, a sub-agent quietly returns a number computed with different logic than the orchestrator assumed, and nobody catches it because the output still looks like a normal answer.

Query-era semantic layers, built for answering one-off questions, aren't enough on their own anymore either. An agent doesn't just query data, it investigates, plans a next step, and acts on what it finds. That means the semantic layer also needs to carry persistent business context, domain rules, and the logic for how an investigation should unfold, not just a dictionary of metric names.

Lineage and auditability when an agent is the actor, not the analyst

Lineage, classically, tracks where a dataset came from and what happened to it along the way. Useful mostly for debugging, reviewed by a human after something already went wrong.

When an agent is the one taking action, lineage has to capture more than the data's path. It has to capture identity (which agent, acting on whose behalf), intent (what decision this data fed into), and the full chain that connects a data access to a specific action taken downstream. An audit log that records only the query, with no record of which agent or which end user triggered it, turns into noise the moment agents start running thousands of queries an hour.

Permissions matter here more than ever. They need to be enforced at the moment of the query, under the actual end user's identity, not waved through under a broad service account that was set up once and forgotten. Lineage stays meaningful only this way when a handful of agents are issuing requests on behalf of hundreds of different people. A subtler problem is that two tables can each be harmless on their own and still produce something sensitive the instant an agent joins them. Classical lineage, built to flag risk at the level of a single field or source, has no mechanism for catching that.

Write actions deserve tighter governance than reads, full stop, because the risk extends to actions taken in the world based on that answer. It's a wrong decision that already executed somewhere downstream. Regulation is catching up to exactly this concern: the EU AI Act took effect in August 2024, with its first obligations (banned practices, AI literacy requirements) landing in February 2025 and the bulk of substantive rules arriving in August 2026. ISO/IEC 42001 and the NIST AI Risk Management Framework, with its Govern, Map, Measure, Manage structure, now expect lineage, bias documentation, and auditability that most existing governance programs simply weren't built for. Gartner predicts 40% of agentic AI projects will be canceled by the end of 2027 over rising costs, unclear value, or weak risk controls, and two of those three causes trace straight back to lineage and auditability gaps.

What an AI-ready data layer actually requires to satisfy all four dimensions at once

Data earns the label "AI-ready" when five things are true at once: it's discoverable, accessible in real time or close to it, governed from source to destination, high-quality and certified, and semantically consistent across every system that touches it. Right now, only around 1 in 10 businesses are considered fully AI-ready because their data is prepared and structured for AI use, even though 78% reported using AI in at least one business function in 2024, up from 55% the year before. Adoption is climbing fast. Readiness isn't keeping pace.

An AI-native data layer pulls structured sources, databases, warehouses, operational systems, into one place without losing the meaning attached to them: shared definitions, entity relationships, constraints, the context that tells a person or an agent how a number is actually supposed to be used.

Mapped against the four dimensions, the architecture looks like this:

• Completeness needs a semantic layer with explicit metric definitions, join logic, and mapped relationships, since an agent can't invent context the schema left out
• Freshness needs a live query layer, Change Data Capture and stream processing instead of static snapshots, so data gets pulled at the moment of the decision, not at the moment of the last pipeline run
• Semantic context needs a governed translation layer exposing table descriptions, metric definitions, and relationships as structured, queryable metadata, not buried in a prompt or scattered across a vector store
• Lineage needs identity enforcement at query time, access permissions scoped per workload and revocable without breaking a shared account, and audit logs that capture identity and intent alongside the data path itself

None of that requires ripping out the warehouse and starting over. The realistic path for most enterprises sits on top of what already exists, exposing one governed interface rather than replacing the infrastructure underneath it. And structured questions, ones with a precise, correct answer, need to go through structured, queryable data. Routing them through a vector database built for semantic retrieval gets you a plausible-sounding answer through the wrong mechanism, when what the question actually needed was a deterministic one.

The SciHorizon-DataEVA framework groups all of this into four categories: Governance Trustworthiness, Data Quality, AI Compatibility, and Scientific Adaptability. Different label, same conclusion. The old five dimensions measured whether data was fit for a person to read. What matters now is whether it's fit for a machine to act on, and those turn out to be two different tests entirely.