AI cost is being measured in the wrong place

Intro

The cloud invoice has excellent manners. It itemizes itself. Opportunity cost does not.

A recent Private Executive Bites table, ten senior data and AI leaders in a private room in Chicago, surfaced a harder question about enterprise AI economics:

What if organizations are getting better at measuring AI spend while still measuring AI cost in the wrong place?

Compute can be priced. Tokens can be counted. Licenses can be allocated. Infrastructure commitments can be modeled. Those numbers matter.

But last week, in Digital Provenance Is the New Trust Layer, we argued that trustworthy AI requires something deeper than a convincing output. Leaders need visibility into origin, transformation, AI involvement, approval, and the history behind an asset.

The same principle applies to economics. If we only measure the final API charge, we are looking at AI cost without its provenance.

What data had to be repaired first? How much human intervention followed? Which workflow had to be redesigned? What happens when nobody clearly owns the outcome? How much capacity is processing information the organization should have retired years ago?

The invoice tells you what the technology consumed. It does not necessarily tell you what the outcome cost.

That distinction also extends an argument we made earlier in The AI Infrastructure Bill Is Coming Due. AI infrastructure has become a genuine capital and operating concern, spanning compute, memory, networks, power, capacity, and vendor commitments.

But even that is only part of the ledger. Apparently AI has discovered the corporate tradition of having more than one cost center.

The most visible AI expense may not be the most consequential one. Or, as Frank Sung of Loop Capital put it across the table: it is not cost per token, it is cost per outcome.

From the table

Private Executive Bites is a small, invitation-only table of senior leaders convened by Data Products around one consequential data or AI question. No panels, no presentations, no selling from the table. Ideas leave the room; attribution does not, unless the person agrees in writing.

This edition draws on the September table in Chicago. The hypothesis on the cards: enterprises may be measuring AI cost in exactly the wrong place. The three questions: where does bad data show up on your AI bill; who should pay to clean it, the team that created the mess or the team that wants to use it; and if you had to cut AI run cost by a third tomorrow, would you fix the model, the data or the process first?

Three guests agreed to be named. Everyone else appears as “one guest”. The full insight article sets out the whole argument, with the costs most AI models leave out.

Executive summary

The enterprise AI cost model becomes distorted when organizations treat infrastructure consumption as the whole cost of AI.

The more useful unit is the completed, quality-adjusted business outcome. That forces a different set of questions.

How much rework was required because the underlying data was inconsistent?

How much processing is being spent on information that should have been archived, reconciled, or removed?

How much human intervention is required before an output becomes usable?

When performance disappoints, does the organization know whether to change the model, improve the data, or redesign the process?

And who owns that decision?

These questions are not entirely new.

An earlier table asked where AI pilots actually break: the model, the data, or the ownership. It also raised the question of who owns a data product when it fails and what must be true before agents are introduced into regulated or customer-facing processes.

The September table pushed that thinking into economics. The issue is not that token cost is irrelevant. It is that token cost is one line on a much larger bill. And some of the expensive lines do not come with invoices.

A cheap inference that produces an expensive workflow is not cheap AI.

Core areas to watch

1. 1

   The meter is not the cost model

   Most AI platforms make consumption increasingly observable.

   That is useful. But it can also create false precision.

   A business can know exactly how many tokens a workflow consumed and still have very little idea what it costs to produce a reliable outcome.

   One guest at the table said it plainly: the cost starts the moment someone writes a bad prompt. If you do not know how to ask, you pay for the answer anyway. Hema Raj Trukenbrod made that point before anyone with a platform budget did.

   This distinction matters because an AI system does not create value merely by responding.

   It creates value when the response survives the rest of the workflow: validation, correction, approval, integration, execution, and ultimately business impact.

   That changes the unit of analysis.

   Instead of asking only: What does this model cost to run?

   Leaders also need to ask: What does this process cost to finish?

   Those are different questions.

   A lower-cost model that creates more exceptions, more manual review, or weaker outcomes can easily be economically inferior to a more expensive model embedded in a better-designed system.

   The cheapest token in the room can still have very expensive friends.

   Key takeaways

   • Track AI economics at the workflow and outcome level, not solely at the model or API level.
   • Include human review, exceptions, rework, latency, and downstream processing when measuring the economics of an AI-enabled process.
   • Compare alternatives on quality-adjusted outcomes rather than consumption price alone.

   Unit economics should end where the business outcome ends, not where the API call does.

2. 2

   Bad data does not stay in the data budget

   Bad data has a habit of traveling.

   Duplicate records, outdated information, inconsistent definitions, and data that has outlived its useful life may begin as data-management problems.

   Once AI starts consuming them, they become compute problems, quality problems, process problems, and eventually decision problems.

   The table produced a particularly useful distinction. Julianna Cole, MBA, PMP, CIO of the National Board of Osteopathic Medical Examiners, argued that the economic cost of poor data is not mainly what it costs to clean or store it. It is opportunity cost: the confident, wrong answer that somebody acts on, and that nobody discovers until a year later. If you are lucky, someone catches it. If you are not, the organization has already moved.

   That includes the value that never materializes because the organization cannot confidently use what sits underneath the AI system.

   The same issue appears with redundant, obsolete, or trivial data. One guest had a name for it: data rot.

   Moving it to inexpensive storage may reduce one line item. Continuing to retrieve, transform, embed, index, reconcile, and process it can simply relocate the expense.

   Storage may be cheap. Teaching AI to repeatedly reconsider information nobody needed in the first place is a less compelling bargain.

   This is why “Who pays to clean the data?” is more than a budgeting question. It is an operating-model question. The table used that question to expose the tradeoff, and split on it. One guest said the creator should pay, then answered herself: how many creators control the budget? Another said the payer is whoever sees the value, which is often how it happens, until the root cause goes unfixed and the bill comes back.

   If the cost appears in one function while the benefit appears in another, an enterprise can rationally underinvest in something everyone agrees it needs.

   Key takeaways

   • Treat data quality as part of AI economics, not as a prerequisite that mysteriously funds itself.
   • Measure the recurring processing cost of poor, duplicated, and obsolete data, not only storage.
   • Make the economic benefit of remediation visible to the functions benefiting from better AI outcomes.

   Bad data rarely sends an invoice. It prefers recurring charges.

3. 3

   Model, data, or process? Fixing the wrong layer gets expensive

   When an AI system underperforms, the model is an attractive suspect. It is visible. It has a version number. Someone can replace it.

   The underlying problem may be elsewhere. A better model will not repair a broken approval process. More training will not resolve contradictory master data. A larger context window will not create ownership where none exists.

   This is also where the provenance argument from last week's issue becomes useful again.

   Digital provenance asks organizations to preserve enough history to understand how an asset became what it is. AI operations need a similar diagnostic discipline.

   If an outcome fails, leadership needs enough traceability to determine whether the problem originated in the model, the data, the workflow, the controls, or the handoff between them.

   Otherwise, remediation becomes expensive guesswork.

   The table was asked to choose: model, data, or process first?

   There was no universal answer. That is probably the point. One guest went for the model, on the grounds that most organizations use the top-of-the-line model for everything, including asking what the weather is. Another went for process, because the process is what creates the data. A third refused to answer until she had seen the detail, which is the most defensible position of the three.

   The useful discipline is determining which layer is actually constraining the outcome before spending money on another.

   Frank Sung, Vice President of Data Management at Loop Capital, offered the table a set of tests for a related question, when to let AI decide at all: impact, reversibility, explainability, and accountability. If the impact is high, the decision cannot be undone, nobody can explain it, or nobody owns it, a human decides.

   They are not the main framework for this issue, but they provide a useful executive pause before “upgrade the model” becomes the organizational equivalent of turning it off and back on again.

   Key takeaways

   • Diagnose performance across model, data, and process before prescribing a technical fix.
   • Separate changes that are easy to reverse from those that materially alter workflows, controls, or customer outcomes.
   • Ask who can explain the change, who owns its impact, and who is accountable if behavior changes in production.

   Not every AI problem needs a better model. Some need a better process. A few need someone to admit nobody owns the process.

4. 4

   Adoption has economics too

   AI adoption is often measured through access, licenses, active users, or feature utilization.

   Those are useful signals. They are not the same as useful adoption.

   The conversation raised a subtler point: organizations can interpret low usage as resistance when employees may never have been shown what effective AI use looks like within their role. More than one guest had met an organization whose entire AI strategy was a license count.

   That distinction matters. It does not mean every adoption problem is really a training problem.

   It does not mean every underused platform deserves rescuing. It means utilization needs context.

   Employees need to understand where AI improves the work, where judgment remains necessary, which tasks are appropriate, what good output looks like, and when the tool should not be used.

   Otherwise, organizations can spend heavily on capability while measuring success through logins. That is a perfectly respectable metric for software access.

   It is a remarkably unambitious metric for transformation.

   There is also a direct connection to last week's provenance discussion. We argued then that a trust signal is only useful when people understand what it means and what action it should trigger.

   The same principle applies to adoption.

   Access to a tool is not the same thing as knowing how to use it well.

   Key takeaways

   • Define good AI usage by role and workflow rather than expecting employees to discover it independently.
   • Measure adoption through changed work and improved outcomes, not availability alone.
   • Distinguish between a tool people have not learned to use well and a tool that simply does not create enough value to deserve adoption.

   Buying the seat is procurement. Changing the work is adoption.

Boardroom implications

Executives do not need to inspect every token invoice. They do need an economic model that reflects how AI actually creates, or fails to create, value.

Five questions deserve a place in that discussion:

1. Outcome. What completed business result are we buying, and what is its quality-adjusted cost?
2. Data. How much AI spend is compensating for poor, duplicated, outdated, or poorly governed information?
3. Intervention. When performance falls short, how do we determine whether the constraint is the model, the data, or the process?
4. Ownership. Who owns the outcome when the costs and benefits cross technology, data, finance, and operating functions?
5. Adoption. Do employees know what effective AI use looks like in their role, and are we measuring changes in work rather than software access?

Infrastructure asks whether the organization understands the physical and financial dependencies behind AI.

Provenance asks whether the organization can understand the history and trustworthiness of what AI produces.

Economics asks whether the organization can trace what it actually costs to create a useful result.

Different problem. Same discipline. Make the invisible dependencies visible before they become expensive.

The wrap-up

Enterprise AI economics is becoming more sophisticated.

Sophistication can still be pointed at the wrong denominator.

Tokens matter.

Compute matters.

Licenses matter.

So do dirty data, unnecessary processing, broken workflows, ownership gaps, manual intervention, and tools that never make the jump from “available” to “useful.”

The goal is not to make every hidden cost perfectly measurable.

That would require its own transformation program, several steering committees, and inevitably a dashboard nobody opens.

The goal is to stop pretending that the easiest costs to meter are automatically the costs that matter most.

The better question is simple:

What does it cost us to produce a reliable business outcome with AI, and where is that cost really coming from?

Once leadership can answer that, AI economics starts looking less like a cloud bill and more like what it actually is: an operating-model decision.

Call to action

1. Reframe. Choose one important AI workflow and define the business outcome rather than the technology activity.
2. Recalculate. Add data preparation, human review, rework, exceptions, and downstream processing to the cost model.
3. Diagnose. When results disappoint, identify whether the constraint is model, data, or process before buying another technical fix.
4. Assign. Make ownership explicit for the outcome, not merely for the platform.
5. Teach. Show people what good AI usage looks like in the work they actually perform.

The AI bill is coming due. Before paying it, make sure you are reading the whole invoice.

Where Data Products stands

Our view is that most AI cost problems are data and operating-model problems wearing a model's clothes. The work that pays is usually unglamorous: knowing your data before you feed it to anything, deciding who owns the outcome before the outcome fails, and measuring what the business got rather than what the platform consumed.

Join a table

Private Executive Bites tables are deliberately small and curated. Tell us your city, industry or function, and the question you think belongs on the table. Requesting an invitation does not guarantee a seat.