On September 24, 2026, Microsoft updated the documentation for a quiet little toggle that cuts Azure OpenAI token costs by half. The feature is called Flex processing, it is in public preview, and the way you turn it on is a single request field: set service_tier to flex and your input and output tokens bill at 50% of the Standard rate (Microsoft Learn). No new deployment, no contract, no capacity commitment. One field.
That is the headline, and it is real. The part that decides whether Flex saves you money or quietly costs you more is everything underneath the headline, and almost none of it is in the launch announcement.
Inference now has a tier menu, and most teams run everything on the expensive one
For a decade, cloud compute taught finance teams a mental model: on-demand when you need flexibility, reserved or savings plans when you can commit, spot when you can tolerate interruption. AI inference spent its first two years with no equivalent. You called the API, you paid the rate, done. That is over. As of late September, a single Azure OpenAI deployment can serve requests across five distinct cost-and-latency tiers, and you pick the tier per request.
| Tier | How you select it | Latency and availability | Cost vs Standard |
|---|---|---|---|
| Flex | service_tier: flex per request |
Variable; can return HTTP 429 when capacity is unavailable; no SLA | 50% discount |
| Standard | service_tier: default (or deployment default) |
Best-effort online | Baseline pay-per-token |
| Priority | service_tier: priority |
Lower, more consistent latency | Priority rate (above Standard) |
| Batch | Submit an async batch job | Targets completion within 24 hours | Discounted batch rate |
| Provisioned (PTU) | Reserve provisioned throughput units | Reserved, predictable throughput | Hourly PTU billing or reservation |
Source: Azure OpenAI Flex processing docs and Maximize your ROI for Azure OpenAI.
Read that table as a market. Priority is the premium lane. Standard is on-demand. Flex is spot: cheap, preemptible, no guarantees. Batch is the deferred-delivery option that has quietly matched the same 50% discount for a while, and Provisioned is the reserved-capacity commitment. The economics are not Azure-specific either. OpenAI’s own Batch API carries a flat 50% discount with a 24-hour turnaround, and its Flex tier mirrors the per-request version. Anthropic’s batch pricing lands in the same place. Three providers converged on the same answer: if you will wait, we will halve the bill.
The problem is that most organizations route 100% of their inference through Standard, because Standard is the default and nobody has been assigned to decide otherwise. The State of FinOps 2026 survey found 98% of organizations now manage AI spend, up from 31% two years earlier, yet managing spend and routing it are different jobs. Watching a bill grow is not the same as moving work to the tier that fits it.
What actually belongs on Flex
The discount only pays if you can tell interactive work from the rest, and in most codebases that line was never drawn.
The Microsoft guidance is specific about where Flex fits: model evaluations, data enrichment, document analysis, and asynchronous application workflows. The common thread is that no human is waiting on the response. A nightly evaluation run that scores last week’s model outputs does not care whether each call takes four seconds or forty. A pipeline that enriches 200,000 CRM records overnight does not care. A document-analysis job that summarizes a backlog of contracts does not care. These are the workloads that were already quietly consuming a large share of many AI bills, billed at full Standard price for latency nobody needed.
Keep the opposite set on Standard, Priority, or provisioned throughput: anything a user is waiting on, anything with a strict latency target, anything your application cannot retry. The classification is not technical. It is a product-and-finance decision about which requests have a human on the other end, and it is the single highest-leverage thing a FinOps practitioner can do with inference spend this quarter.
A rough sense of the prize: if a team spends 40,000 dollars a month on Azure OpenAI and even half of that is non-interactive evaluation, enrichment, and offline analysis, moving that half to Flex is a 10,000-dollar monthly line item recovered, before touching a single prompt. Prompt caching stacks on top, because eligible cached input tokens keep their cached-token discount on Flex, which is a reminder that the write-versus-read economics of caching still apply underneath the tier choice.
The 50% has an asterisk, and it is operational
Here is where the launch note stops and the engineering bill starts. Flex is preemptible capacity, and Microsoft is unusually blunt about the consequences.
Flex has no latency SLA and no service SLA. A request can return HTTP 429 when Flex capacity is not available, and the docs note that preemption is more likely during peak hours. There is no automatic fallback to Standard: if Flex is full, your request fails, and your application has to explicitly retry with service_tier set to default if it wants the job done. The moment it does that, the request bills at full Standard price, so every fallback erases the discount it was chasing. Flex and Standard also share the same quota on the Global Standard deployment, so a burst of Flex retries can starve the interactive traffic sitting next to it.
There are sharper edges. Flex has limited model support at launch; gpt-5.6-sol (version 2026-07-09) is the first and, at preview, essentially the only supported model, and an unsupported model returns HTTP 400 rather than falling back. Flex requests can simply take longer, so Microsoft recommends a client timeout around 15 minutes and bounded exponential backoff with jitter. None of that exists for free. Someone has to write the retry logic, the fallback policy, the timeout handling, and the monitoring, and that engineering time is a real cost that has to be set against the 50% saving. For a workload that is genuinely delay-tolerant and runs at volume, the math is easy. For a trickle of a few thousand calls a month, the build cost swamps the saving and you should leave it on Standard.
Why this lands on the FinOps desk, not just the ML team
The reason inference tiering belongs in a FinOps conversation is that the saving is invisible unless someone makes it visible and someone decides the routing policy. Left alone, engineers optimize for the thing they feel, which is latency, and latency always argues for the faster, more expensive tier.
Running operations at a large telecom for two decades, I watched the same pattern with batch windows: the cheap overnight capacity sat half-used because no one owned the decision to move non-urgent jobs into it, and every team defaulted to the daytime lane that felt safer. Inference tiering is that decision again, with a token meter attached. The fix is the same: make it a policy, not a preference.
The encouraging part is that Azure made the accounting tractable. Flex usage bills on dedicated Flex meters, so in Cost Management you can filter Cost Analysis by meter and by deployment tag and see exactly how much moved to Flex, how much fell back to Standard, and what the net saving was after fallback. That turns the tier choice into something you can report on, the same way you would report commitment coverage for reserved instances versus savings plans. Track the 429 rate and the fallback rate as first-class signals; a Flex workload that falls back half the time is not a Flex workload, it is a Standard workload with extra steps.
This is also where the tiering model connects to the broader shift in how AI cost is measured. Routing work to the right tier is a per-request version of the cost-per-outcome question that CFOs keep raising, and it is one more piece of the AI estate that stays dark until finance and engineering look at it together, the same visibility gap that turns unmanaged AI usage into a budgeting problem.
The practical read
Azure OpenAI Flex is not a trick and not a trap. It is a spot market for tokens, and like every spot market it rewards the buyer who knows which of their jobs can wait and punishes the one who routes everything to it blindly. Classify your inference into interactive and background. Put the background work on Flex or Batch with real retry and fallback logic. Keep the user-facing work on Standard, Priority, or PTUs. Watch the dedicated meters, and measure the net saving after fallback rather than the sticker discount. The 50% is sitting there in a single request field. Whether you actually keep it depends on a decision no API will make for you.