Definition
For any kind of work there is a cheapest model whose answer actually holds. Routing to it saves real money; routing below it produces answers that fail quietly and cost more after the redo. The goal is the cheapest model that produces the desired outcome for that kind of work.
The waterfall page showed the routing decision. This page is about what it is worth. Our specimen reply (three sentences about a refund) could have been written by a small, mid, or frontier model tier, at wildly different prices. The question a gateway has to answer thousands of times a day is which tier each piece of work deserves, and both of the obvious answers turn out to be wrong.
How big the model tier price gap is at volume
The gap between tiers is not a rounding error, and it compounds with every request. Slide the volume below to your organization’s scale and read the monthly bills.
At 10k requests a day, the same month of traffic costs $120 on the small tier and $5,400 on the frontier tier: a 45x gap that scales with every request. This is why "route everything to the cheapest" is so tempting. Hold that thought.
At any real volume, that gap is the strongest argument anyone will ever make for “just route everything to the cheapest model.” It is also, and this is the whole page, only half of the ledger. The other half is whether the answer actually held.
Why cheapest-everything and frontier-everything routing both fail
An outcome, for our purposes, is the thing you actually wanted: a reply that ships without a human touching it, a fact that is right, a reconciliation that balances. Prices are per request, but value is per outcome, and the two come apart the moment a cheap answer fails and a person has to redo it. The figure below runs one day of the specimen’s mixed workload (100 requests: quick facts, drafts, hard reasoning, sensitive data) under three rules. Run all three and compare the effective cost rather than the sticker price.
The workload: 100 requests, 25 each of the four kinds from the specimen's day (quick facts, drafts, hard reasoning, sensitive data). Pick a rule to run it.
One workload, three rules. The two blanket rules each fail differently. Run all three.
Read what happened. Cheapest-everything posted a tiny model bill, then handed half its answers back to humans, and the redo premium made it the expensive strategy. Frontier-everything held every answer and paid roughly triple for the privilege, and it still committed the one unforgivable error: a price rule has no idea what policy is, so it shipped sensitive data outside. Each blanket rule fails in its own direction, which is precisely the cost-quality tradeoff in its honest form: a per-kind question with a different right answer each time, rather than a single knob between cheap and good.
The cheapest model that holds, per kind of work
So the winning rule is: for each kind of work, the cheapest tier whose answers actually hold, with policy holding the veto. The figure below runs that rule in front of you. The same requests stream through all three tiers at once; green dots land in the shipped bin, red dots reach the model just fine and then bounce to the human-redo pile, and policy walls block lanes the data may not use. Change the kind of work and watch the ✓ move.
draft a reply · The small model's draft got rewritten by a human, so it was not cheaper. The mid-tier model is the cheapest one that actually held. Two challenges: find the work where the frontier tier is the cheap choice, and the one where price never got a vote. Figures are illustrative.
Take the two challenges in the caption. The work where the frontier tier is the cheap choice is hard reasoning, because every cheaper tier’s dots bounce, and a bounced answer costs more than the most expensive model that holds. And the work where price never got a vote is the sensitive kind, where the wall decides before the wallet does. Once you have seen both, you have the whole lesson: knowing which tier holds, for which kind of work, is exactly the knowledge a gateway accumulates by watching outcomes, and it is worth real money every day.
The last page in the series asks what the gateway can see at all, which is where that knowledge comes from. For the reference treatment of the spend knob, see cost-aware model routing; for learning “what holds” from your own history, see evidence-based routing.