Forecasting is now the capability FinOps teams most want to build. In the State of FinOps 2026 report, 98% of practitioners say they manage AI spend, up from 63% a year earlier, and forecasting sits near the top of the skills they plan to add. Here is the uncomfortable part almost nobody says out loud: the moves that cut your AI bill are the same moves that make your forecast wrong. The better your team optimizes AI spend, the less your forecast is worth.
That is not a knock on forecasting. It is a description of how AI cost dynamics actually behave, and it explains why the cloud budgeting playbook you spent five years refining keeps missing on AI line items by 30% or more.
The cloud forecasting playbook assumes stable unit economics
Traditional cloud forecasting works because the underlying unit price barely moves. An m6i instance costs roughly what it cost last quarter. An S3 GB is a known rate. You take historical consumption, apply a growth curve, layer in known events (a product launch, a migration), and you land within a few points of actual. Reserved Instances and Savings Plans exist precisely because that stability holds: you commit to a baseline you are confident will still be running in twelve months, and you get a discount for the certainty.
AI breaks the two assumptions that make this work. The unit price is not stable, and the consumption is not provisioned.
The FinOps Foundation’s working group on the effect of optimization on AI forecasting put the first problem plainly. Model pricing trends “show costs tend to decrease while quality and performance make significant jumps every 6 to 9 months.” That single sentence should worry anyone building a four-quarter AI budget. If the price of the intelligence you are buying falls on a 6-to-9-month cycle, your historical spend per token is a decaying reference point, not a baseline. A forecast built on last quarter’s rate is overstated the moment a cheaper, better model ships.
The second problem is the unit itself. In AI workloads the fundamental unit is the token, not a compute hour or a provisioned GB. Token consumption scales with behavior: how often a model is called, how long the prompts run, which tier answers the request, whether the output was cached or regenerated. None of that maps cleanly to a commitment you can reserve. You cannot buy a Savings Plan on prompt length.
AI costs move in step-changes, not trends
Here is what actually makes AI forecasting fall apart. Cloud costs drift; AI costs jump. When your team ships an optimization, the bill does not bend, it drops off a cliff, and every projection built before that day is now fiction.
Three well-documented techniques show the size of those cliffs:
Model routing. RouteLLM, the open-source router from UC Berkeley and Anyscale published at ICLR 2025, dynamically sends easy queries to a cheap model and hard ones to a strong model. In the published benchmarks it cut cost by more than 85% on MT-Bench while keeping 95% of GPT-4-level quality. If your team deploys routing mid-quarter, the workload that cost $40,000 in June might cost $6,000 in August. Your annual forecast for that workload is now off by a factor of six, and you did nothing wrong; you did something right.
Prompt compression. Microsoft Research’s LLMLingua compresses prompts before they hit the model, keeping only the informative tokens. It reaches 2x to 5x compression on instructions and 5x to 20x on retrieval context, with reported quality loss under 2%. One documented customer-support deployment fell from $42,000 a month to about $2,100. That is a 95% cut from a single engineering change that touched no business logic.
Caching and deduplication. The FinOps Foundation paper cites a financial-services case that reached a 99% reduction in ongoing token usage by hashing and differentially caching more than 60,000 IAM policies instead of re-sending them. Ninety-nine percent. There is no growth curve that survives a 99% base reduction landing in the middle of it.
Now compare that to the hardware-side numbers in the same paper: a 14% cost reduction on an EMR workload, 15% to 20% on a custom application, an expected 15% on a Kubernetes workload. Those are meaningful, and they look like normal cloud optimization. But the software and consumer-side moves (routing, compression, caching) operate an order of magnitude larger. That gap is the whole problem. Your forecast has to anticipate optimizations that can each remove 85% to 99% of a line item, arriving on an unpredictable schedule, decided by engineers who are not looking at your model.
The commitment mental model finance trusts does not map to tokens
Most finance leaders got comfortable with cloud because FinOps gave them a lever they understood: coverage. Cover 70% to 90% of steady compute with commitments, and you convert variable spend into something close to a fixed, discounted line. The State of FinOps 2026 data shows teams can reach that band with basic forecasting and occasional adjustments.
That lever barely exists for token-based AI spend. You are not reserving capacity you will predictably consume; you are paying per unit of behavior against a price that the vendor may cut next quarter. Committing hard to a twelve-month AI baseline is the one thing you do not want to do, because the odds that a cheaper model or a routing change makes that commitment look foolish are high. The reservation and spot-instance frameworks that govern predictable infrastructure, as the FinOps research notes, do not govern bursty, unpredictable inference efficiently.
I spent years forecasting capacity in Saskatchewan telecom, where the discipline was to project minutes and circuits against a fairly stable per-unit economics. The forecast was hard, but the ground did not move under it. AI forecasting is the opposite situation: the per-unit cost is falling while your team is simultaneously cutting the number of units it takes to do the same job. When both the price and the quantity are dropping on you, a point estimate for next March is not a forecast, it is a guess with a decimal place.
Forecast the unit and the value, not the line total
If the total dollar figure is unforecastable, stop trying to forecast it precisely, and forecast the things that actually behave. Three shifts help.
Forecast cost-per-unit-of-work, then range the volume. The FinOps Foundation’s central recommendation is to establish a clear link between cost-per-unit-of-work and expected business value, so the value AI generates stays proportionate to what it costs. Cost per resolved support ticket, per generated document, per completed agent task: that unit is more stable than the aggregate bill, and it is the number that tells you whether an optimization helped or just shifted spend around. Track it, and a routing deployment shows up as a step down in unit cost that you can explain to a CFO, rather than a mystery drop in a total.
Produce scenario bands, not point estimates. Give the budget three tracks: a no-optimization track (what this costs if nothing changes), an optimized track (what it costs if planned routing, caching, and compression land), and a model-price track (what it costs if the vendor cuts rates on the usual 6-to-9-month cadence). The spread between those tracks is the real information. It tells leadership how much of the forecast is under your control versus the vendor’s, and it stops a single number from being treated as a promise.
Re-baseline every quarter, on purpose. A stale AI baseline is worse than no baseline, because it carries false confidence. Bake a scheduled re-forecast into the quarter, tied to model releases and shipped optimizations, and treat a large favorable variance as a signal that your engineers are doing their job, not as a planning failure. This is where AI budget overruns usually come from: not runaway spend, but forecasts that were never designed to move.
None of this is a reason to skip forecasting. It is a reason to change what you point the forecast at. The teams getting this right have quietly stopped promising a number and started promising a unit economics story: here is our cost per unit of work, here is where we expect it to go as we optimize and as prices fall, and here is the volume range that turns that into dollars.
The scaled-content version of an AI budget is a spreadsheet with a confident total for next December. The honest version is a band, a unit cost, and a standing agreement that the number will move, hopefully downward, and probably faster than anyone upstairs expects. If your AI cost management practice can explain the movement instead of pretending it will not happen, you are ahead of most of the 98%.
For the mechanics underneath this, the hidden markups on every token, the gateway layer that makes routing possible, and the broader FinOps-for-AI framework are the places to go next.
Sources: FinOps Foundation, Effect of Optimization on AI Forecasting; State of FinOps 2026 Report; RouteLLM: Learning to Route LLMs with Preference Data (arXiv 2406.18665); Microsoft Research, LLMLingua.