There is a story making the rounds among enterprise architects right now, and it should be required reading for every CFO about to sign off on an agentic AI project.
A team builds a proof of concept for an AI-powered workflow. The pilot runs for a month and costs $500 in OpenAI API usage. The results look strong. Leadership green-lights production. The team deploys to the full user base. The first month's bill arrives: $847,000. That is a 717-times cost increase from pilot to production. No ROI calculation survives that kind of jump.
The example comes from a case study published in an analysis of enterprise LLM deployments, and it is not an outlier. Another team reported a $50 pilot scaling to approximately $2.5 million per month at full production volume. Real-world enterprise LLM usage has been documented producing monthly bills in the $500K to $1M range. These are not horror stories. They are the pattern. And they explain why so many agentic AI initiatives quietly die in the budget review that follows their successful pilot.
Every interaction with a large language model is priced in tokens. A token is roughly three to four characters of text, or about three-quarters of a word. Commercial AI providers like OpenAI, Anthropic, and Google charge per million tokens, with separate rates for input tokens (what your system sends to the model) and output tokens (what the model sends back).
For generative AI use, where a human types a prompt and reads a response, token costs are modest. A few pennies per interaction. Add it up across a month of employee usage and you end up with a manageable line item.
Agentic AI is a different animal entirely. A Gartner analysis published in early 2026 found that agentic workflows consume between five and thirty times more tokens per task than a standard generative AI chatbot. An agentic system, working through its perceive-reason-act-evaluate loop, may trigger ten to twenty separate LLM calls to complete a single user-initiated task. Multiply that by hundreds of concurrent users and thousands of tasks per day, and the token math starts looking very different from the pilot math.
Five structural factors explain why agentic token costs balloon in production.
The reasoning loop itself is the biggest driver. Each iteration through the agentic loop triggers at least one LLM call, often several. Complex tasks can require dozens of iterations before the agent is satisfied with the result.
Context accumulation is the second. Agents pass context forward between steps. Every tool output, every retrieved document, every intermediate decision becomes input tokens for the next call. A task that starts with a 500-token prompt can end up processing 50,000 tokens of accumulated context by the time the agent finishes.
Tool descriptions add a surprisingly large fixed cost. When an agent has access to many tools, the descriptions of every tool are sent to the model on every call. Teams with hundreds of tools integrated can consume hundreds of thousands of tokens in tool definitions alone before any actual work happens.
Reasoning tokens are a newer cost that most finance teams have never heard of. Modern models with extended thinking capabilities — OpenAI's o-series, Claude's extended thinking mode, Gemini's thinking mode — consume tokens for internal reasoning that never appears in the visible output. You still pay for those tokens.
Multi-agent overhead multiplies all of the above. A system with five specialized agents coordinating on a task incurs context, tool descriptions, and reasoning costs for each of them. Token consumption compounds.
The CFO question is no longer "what does this tool cost per month?" The right questions are different. What is the expected token burn per task? How many tasks will run per day at full volume? Which tasks should route to a cheaper, smaller model and which require premium reasoning? What is the caching strategy for repeated context?
This is a new financial discipline. The industry has started calling it FinOps for AI or AI cost management, and it borrows from the cloud cost management practices that matured over the last decade. Like cloud costs, agentic AI costs are variable, non-deterministic, and driven by usage in ways that traditional SaaS pricing is not.
Three practical tactics separate projects that scale from projects that die at the budget review.
First, implement semantic routing. Not every query needs a frontier model. A routing layer that classifies query complexity and directs simple queries to cheaper, smaller models can cut costs by sixty percent or more without hurting quality on the queries that matter. A standard support ticket does not need the same model that drafts your legal briefs.
Second, cache aggressively. If your agent always starts with the same system prompt or knowledge base, prompt caching can reduce input costs by approximately ninety percent and latency by seventy-five percent. Major providers now support prompt caching natively. The math works immediately.
Third, set hard limits at the application level. Every production agent should have a per-user or per-task token ceiling and an automated kill-switch for anomaly detection. If an agent starts looping or hallucinating tool calls, the ceiling stops it before the bill arrives.
Per-token prices are falling consistently. Epoch AI's analysis of state-of-the-art model benchmarks confirms per-token inference prices have fallen between nine and nine-hundred times per year for various performance milestones, with Gartner forecasting a further ninety percent cost reduction by 2030. This is the cell-phone-minutes story playing out on fast-forward.
But the cost per token is not the cost that matters. The cost that matters is the cost per successfully completed business task, and that cost is driven by volume, architecture, and discipline. Waiting for prices to come down is a strategy that ends with you explaining the $847,000 bill to your board.
Agentic AI is going to produce extraordinary value for organizations that understand the economics before they deploy. It will produce extraordinary bills for organizations that treat a successful pilot as proof that the math works. The difference between those two outcomes is not the technology. It is the discipline you bring to the budget.