If you are trying to work out how to measure AI ROI, Uber’s COO Andrew Macdonald gave the clearest answer by accident, in a May 2026 Rapid Response interview that most companies are quietly living but nobody says out loud. The business had burned through its entire AI coding budget in four months. His conclusion: higher token usage did not translate into a proportional increase in useful consumer features. The link between spend and value, he said, was genuinely hard to draw.
Most companies are in the same position. Their dashboards show adoption climbing, spend growing, prompts running. None of those numbers answer the question that matters: compared to before AI, what actually got better?
The reason is structural. A token-usage dashboard is a school attendance record. It tells you whether students showed up. A school with 98% attendance and no learning outcomes is not a good school — it is a very good roll-call system. Most AI programmes work exactly the same way.
The fix is not a better dashboard. It is running the sequence in the right order.
The short version
- Token dashboards measure activity, not outcomes. Spend climbing is not value growing.
- The correct sequence is literacy, then adoption, then ROI — in that order. Most companies have inverted it.
- Each step has a concrete test. If you cannot pass the test for step one, step two’s numbers are meaningless.
- ROI must be measured against a pre-AI baseline. Not tokens consumed. Not hours saved as self-reported.
- A metric that only exists because AI exists cannot tell you whether AI worked.
The $1,000 version
A COO at a company roughly the size of a serious Australian SME set a KPI around AI token usage: spend per engineer. The usual instinct once a number starts climbing and nobody can explain why.
He spent $1,000 in API costs solving two issues. On a dashboard, that looks like adoption working. But look closer. The $1,000 was on top of his own time: prompting, re-prompting, checking, correcting. The task took roughly the same time it would have taken without AI. He used it anyway, because the company needed the metric to move.
The spend was not buying capability or productivity. It was buying attendance records at API prices. The roll-call looked perfect. Nobody asked whether the class had learned anything.
This is not an isolated case. According to Writer’s 2026 Enterprise AI Survey, 59% of enterprises invest at least $1 million a year in AI. Only 29% report significant ROI. PwC’s 29th Global CEO Survey found just 12% of CEOs could identify both reduced costs and grown revenue from AI in the past twelve months.
The gap between spenders and earners is not technology. It is sequence.
The sequence that actually works
The order matters. Almost no one follows it.
Step 1: Literacy
What it means:
The people using AI know how to use it well. Not whether they use it — whether they use it well. A team that prompts poorly, cannot verify outputs, and does not know when AI is the wrong tool is a spending team, not a capable one.
Why it comes first:
A team with low AI literacy spending confidently looks identical, on a dashboard, to a team with high AI literacy spending confidently. The dashboard cannot tell them apart. This is the same problem software engineers faced a generation ago when they were paid by lines of code written. The engineer who solved a problem in 50 lines got penalised against the one who wrote 500. Precise prompting uses fewer tokens. The dashboard rewards volume. It cannot see quality.
How to test it:
Pick one task your team runs regularly. Run it with AI. Compare output quality and time taken against your pre-AI baseline. Three questions:
- Did the AI-assisted output meet the same quality bar as the manual version?
- Did it take materially less time, accounting for prompting and verification?
- Did the person doing it feel in control of the output, or were they hoping it was right?
If the answer to any of these is no, literacy is the constraint. Do not move to adoption metrics until this passes. A skills session, a prompt library, or a structured review of how the team is actually using the tools will do more than any dashboard.
Step 2: Adoption
What it means:
Once literacy is real, adoption tells you how much of the potential is being captured. Before that point, adoption is spend with better attendance records.
Why it comes second:
High adoption across a low-literacy team is a cost problem disguised as a progress metric. You are paying for scale before you have proven the thing scales well. The number goes up. The outcome does not follow.
How to measure it meaningfully:
Adoption only means something when measured against a specific workflow with a known literacy baseline. The question is not “what percentage of the team used AI this week.” The question is “what percentage used AI well on the tasks where we know it should help?” Two things to track once literacy is established:
- Workflow coverage: For the tasks where AI has been proven to work, what percentage are being run through it consistently?
- Reversion rate: How often do people complete a task with AI and then redo it manually? Reversion is the most honest signal that literacy is not yet there, regardless of what the adoption figure says.
If reversion is above 20% on a given workflow, treat that as a literacy problem, not an adoption problem. More prompting to use the tool is not the fix.
Step 3: ROI
What it means:
A measurable change in a business metric that existed before AI did. Cycle time. Error rate. Revenue per person. Customer response time. Defect rate. Not tokens consumed, not prompts run, not hours saved as self-reported.
Why it comes last:
A metric that only exists because AI exists cannot tell you whether AI worked. The comparison must be to before — which requires having recorded what before looked like.
How to measure it:
Three steps.
Set the baseline first. Before any new AI rollout, record current performance on the workflows you intend to change. This takes an afternoon. Reconstruct from historical data if you missed it: project management tools, email timestamps, invoicing records, support ticket logs.
Run a 30-day comparison. Once AI is operating on the workflow, measure the same metrics over 30 days — not a new set of metrics designed to make the comparison easier. Compare directly.
Be honest about confounding factors. If the team changed their process at the same time as adopting AI, you cannot cleanly attribute the outcome. Note the confounds and account for them.
If the outcome metric improved, you have a case for continued investment. If it did not, the constraint is usually literacy, not the tool. Buying more access before addressing literacy does not fix the problem.
This is where most AI automation investments quietly disappoint — not because the tools do not work, but because the measurement framework was wrong before the first prompt was written.
How to measure AI ROI in practice: a 90-day plan
For a business starting from scratch, here is what running the sequence actually looks like.
Days 1 to 30 — Literacy sprint. Pick one team, one workflow. Run the literacy test. Identify gaps. Run one targeted training session or build a shared prompt library for that workflow. Re-run the test. Do not move forward until the team passes it.
Days 31 to 60 — Adoption measurement. With literacy established on that workflow, measure adoption and reversion. Set a target: 80% consistent adoption, under 20% reversion. If you hit it, the workflow is ready for ROI measurement. If not, go back to literacy.
Days 61 to 90 — ROI measurement. Compare the 30-day AI-assisted performance against the pre-AI baseline on your chosen metric. Document the result. Use it as the template for the next workflow.
At 90 days, you have one workflow with a defensible ROI number and a repeatable process for the next one. That is more useful than six months of token dashboards across the whole business. If you want a structured approach to this across your organisation, our technology strategy service is built around exactly this kind of diagnostic.
What Uber’s problem actually was
Uber’s problem was never that the budget grew. Budgets for capable things grow.
The problem is not having an answer, four months in, to the older and far less exciting question: compared to before, what actually got better, and by how much?
Andrew Macdonald’s test was exactly right: how many projects on the cutting room floor got moved forward because AI accelerated the engineering work? The answer, he said, was hard to draw a clean line to — even when token usage was trending astronomically.
Counting tokens spent is not the same as counting problems solved. The dashboard has perfect attendance. It just cannot tell you whether the business got smarter.
If you want to run this diagnostic with support, start with 20 hours free.
Common questions
AI ROI is the measurable change in business performance attributable to AI adoption, expressed against a baseline that existed before AI was introduced. It is calculated by comparing pre-AI metrics — cycle time, error rate, revenue per person, response time — against the same metrics after AI has been running at scale. Token spend, adoption rates, and hours saved as self-reported are activity metrics, not ROI metrics.
Token usage tells you what AI cost, not what the business gained. A team with low AI literacy spending confidently produces the same token dashboard as a high-literacy team. A high-literacy team will often use fewer tokens because they prompt more precisely — meaning they look worse on the dashboard despite performing better. ROI requires an outcome metric with a pre-AI baseline, not a cost metric without one.
Before AI adoption, record the baseline performance on the workflows you intend to automate or augment. Relevant metrics depend on the workflow: processing time, error rate, throughput per person, cost per unit, customer response time. Capturing these before adoption is what makes an honest ROI comparison possible later.
AI literacy is the practical capability to use AI tools effectively: knowing how to write a useful prompt, when AI is the wrong tool, and how to verify outputs. It comes before adoption metrics because adoption without literacy produces spend, not capability. A team using AI poorly at scale costs more than a team not using it at all, and both look identical on an adoption dashboard.
Enough time for the workflow to stabilise and for the comparison period to be equivalent to your baseline period. For most operational workflows, 30 to 90 days of consistent AI-assisted operation is the minimum for a credible comparison. Measuring at week two, before the team has developed genuine proficiency, produces results that are neither representative nor defensible.
Pause new adoption and run a diagnostic on one workflow. Reconstruct a pre-AI baseline from historical data. Measure the same workflow with AI over 30 days. If the outcome metric improved, you have a case for continued investment. If it did not, the constraint is likely AI literacy, not tool capability. Buying more access to the tool before addressing literacy does not fix the problem.