The question on the card
Every seat had the same card. It put a hypothesis to the table, that enterprises may be measuring AI cost in exactly the wrong place, and three questions. Where does bad data show up on your AI bill? Who should pay to clean it: the team that created the mess, or the team that wants to use it? And if you had to cut AI run cost by a third tomorrow, would you fix the model, the data or the process first?
The invoice is the easy place to look. Compute is priced, tokens are counted, licenses are allocated, and every platform reports consumption in more detail each quarter. Those numbers matter. But as Frank Sung, Vice President of Data Management at Loop Capital, put it to the table, the number that matters is not the cost per token. It is the cost per outcome.
Start with the unit
The unit is a completed, quality-adjusted outcome. Not a response, but a response that survived the rest of the workflow: checked, corrected where it needed to be, approved, acted on, and right. Measured that way, a cheap model that throws off more exceptions can cost more than an expensive one inside a better process, and a workflow that looks efficient on the invoice can be the most expensive thing a team runs.
The cost also starts earlier than most models assume. Hema Raj Trukenbrod made the point before anyone with a platform budget did: it starts the moment someone writes a bad prompt. If you do not know how to ask, you pay for the answer anyway. Another guest made the same point about data. Hand a model a table with hundreds of columns and ask it to analyze everything, and you get a large bill and a vague answer. Point it at the few data elements that decide the question, and both improve.
A cheap inference that produces an expensive workflow is not cheap AI.
The ledger most AI cost models leave out
Here is the bill as the table described it, line by line. Only the first line arrives with an invoice.
| Line | Where it shows up | Who usually pays |
|---|---|---|
| The meter | Tokens, compute, licenses and reserved capacity | The platform budget |
| Preparation | Cleaning, reconciling and reshaping data before AI can use it | The data team, often unbudgeted |
| Rot | Processing duplicate, outdated and unused records that should have been retired (see the August table) | Everyone, without seeing it |
| Review | People checking, correcting and approving outputs before anyone uses them | The business team, in time |
| Rework | Exceptions, reruns, escalations and fixes downstream | Operations |
| Wrong decisions | Confident answers acted on before anyone notices they were wrong | The whole organization, late |
| Unused capacity | Seats and licenses that never change how the work is done | A budget line nobody revisits |
Julianna Cole, CIO of the National Board of Osteopathic Medical Examiners, argued that the line that matters most never appears on any statement: opportunity cost. Poor data does cost money to clean and store, but the real damage is the confident, wrong answer that someone acts on and nobody discovers for a year. If you are lucky, someone catches it. If you are not, the organization has already moved. She added a quieter cost: every bad result hands a skeptic a reason not to adopt something the organization needs.
Another guest framed the same risk as speed. AI lets an organization reach decisions faster, which is an advantage only if the data is right; wrong data delivered quickly means wrong strategic decisions made quickly. The same guest named two costs that rarely make it into the model at all: rebuilding trust after a visible mistake, and teaching people what good use looks like.
One story made the point concrete. A team was handed an AI-built deliverable by outside advisers: a spreadsheet ranking where to focus the business, on two tabs that used slightly different methods and did not agree, with nothing to show how either had been checked. The deliverable was cheap to generate. Checking it was not, and nobody had priced that.
Who pays to clean it?
The second question split the table, which was the point of asking it. One guest said the team that creates the data should pay, then answered herself: how many creators control the budget? Julianna Cole’s view was that whoever sees the value pays, which is usually how it happens, until the root cause goes unfixed and the same bill comes back. Others pointed to the accounting: whether cleanup counts as a capital project or an operating expense changes who is willing to fund it.
Underneath the disagreement is an operating-model problem. When the cost of fixing data lands in one function and the benefit in another, an organization can rationally underinvest in something everyone agrees it needs. Several guests had seen data teams moved under finance, where they are managed as a cost. Our own view, offered at the table, is that data and technology leaders do a poor job of showing the revenue their work makes possible, and that this is theirs to fix.
In practice, we charge the cleanup to the outcome it enables, and make that outcome’s owner the person who decides on it. A cleanup with no owner downstream is a cleanup that will need doing again.
Model, data or process first?
Asked where to cut a third of run cost, the table did not agree, which is probably the most useful result of the evening. One guest would start with the model, because most organizations use the most capable model for everything, including questions a far smaller one answers as well. Another would start with the process, because the process is what creates the data. A third would not answer until she had seen the detail, which is the most defensible position of the three.
The discipline is to find the layer that is actually constraining the outcome before paying to change any of them. A better model will not repair a broken approval process. More training will not reconcile contradictory master data. Without enough traceability to see where an outcome failed, in the model, the data, the workflow or the handoff between them, every fix is an expensive guess.
Frank Sung offered the table four tests for a related question, when to let AI decide at all: impact, reversibility, explainability and accountability. If the impact is high, if the decision cannot be undone, if nobody can explain it or if nobody will own it, a person decides. The last test carried the most weight around the table, because a model cannot be held accountable for anything.
Adoption has a cost line too
More than one guest had met an organization whose entire AI strategy was a license count. Seats bought is procurement; it says nothing about whether the work changed. Low usage is often read as resistance when nobody has shown people what good use looks like in their role. And some organizations, the table noted, treated AI as a playground with no baseline, then found the budget spent within weeks because nobody was watching consumption.
Adoption deserves the same treatment as cost: measure it by changed work and better outcomes, not by access. Adoption & Workforce sets out how.
Measuring it properly, in one workflow
- Pick one workflow that matters. Name the business outcome it produces, not the technology activity.
- Trace a sample end to end. Follow real outcomes from the data they read to the decision they informed, and note every human touch on the way.
- Price the hidden lines. Add preparation, review, rework and exceptions to the meter, and estimate what a wrong answer costs once someone acts on it.
- Find the constraining layer. Decide whether the model, the data or the process limits the outcome before changing any of them.
- Name the owner of the outcome. Not the owner of the platform: the person who answers for the result and controls the budget to fix it.
None of this needs every hidden cost to be measured perfectly; that would be a transformation program of its own. It needs leaders to stop assuming that the easiest costs to meter are the ones that matter most.
Where to start
- Value & EconomicsThe value of an outcome, what it costs to build and what it costs to run, in one view.
- A Data Readiness reviewWhether the constraint is the model or the data underneath it.
- Building the AI business case a CFO will fundThe number that decides funding, and the run cost most cases leave out.
