What drives agentic AI token costs?
Most API models charge by the token, for what you send in and what the model writes back. An autonomous agent thinks in a loop: it reads the task, plans a step, calls a tool, reads the result, and sends all of it back to the model to decide what's next. Every pass resends the instructions, the growing context and the latest tool output. One request is cheap. Across millions, those repeated round trips are where the bill grows, and because the loop sets its own length, the total is hard to forecast.
Why do routine steps not need a model?
A lot of enterprise work is routing, data synchronization and approvals: move a record, check an amount against a limit, notify the right person. That work doesn't need a model to reason about it; it needs rules and a connection to the right system. When an agent sends it to a model anyway, you pay inference prices for something a rule would do for free. A workflow-first design starts from the process itself and reserves inference for the steps that need interpretation, which is where the paper says the savings live.
An illustrative cost scenario (not a benchmark)
The white paper models one million transactions a month two ways:
- Agent-led: the model runs on 100% of transactions, at 12,000 input and 3,000 output tokens each.
- Workflow-led: the model runs on 20% of transactions, at 3,000 input and 750 output tokens each.
- Claude Sonnet 5.5 list price, November 2026: $2 per million input tokens and $10 per million output tokens.
At those assumptions, the agent-led design models to about $54,000 a month and the workflow-led design to about $2,700, a difference of roughly $51,300 a month, or $615,600 a year, in model inference alone. The 20x gap is two choices multiplied: running the model on a fifth of the transactions (5x) and using a quarter of the context per call (4x).
Treat this as a reason to measure before you commit. Real costs shift with model choice, caching, batching, retries, context size and service levels, and the scenario leaves out licensing, hosting, integration and human review, so it doesn't prove a lower total cost of ownership by itself.
The paper runs the same workload across a few setups to show how much the levers matter:
- The same agent with 9,000 of its 12,000 input tokens cached: about $36,900 a month.
- The same agent on Claude Haiku 5.5: about $2,700, the same as the workflow on Sonnet.
- The workflow on Claude Haiku 5.5: about $135.
A well-optimized agent narrows the gap, and the same levers, caching, batching and smaller models, also apply inside a governed workflow.
There's a second benefit beyond the headline number, and it's predictability. The white paper notes that cutting unnecessary inference improves predictability as transaction volumes rise, which is what matters when you're defending a budget rather than running a pilot. Its architectural comparison draws the same line: an agentic design may require repeated model turns, while a governed workflow can restrict AI to selected tasks. Reserve the model for the few steps that need it and the cost per transaction stops drifting with every change in prompt length or loop depth, so finance can plan against a number that holds. The paper is equally firm on proof: a defensible claim needs a head-to-head test of the same workload, the same integrations, the same security requirements and comparable service levels, measured against a well-optimized agent that already uses caching and an appropriate model.
To see where your own agentic AI token costs would land, map one high-volume process: which steps really need a model, which can run on rules, and what each would consume. An estimate on your own workload is worth more than any illustrative figure.
Keep reading: Governed orchestration vs. autonomous agents · Intelligence is not authority



