Use the cheaper model most of the time

Teams tend to pick one model, wire it to everything, and then discover the bill. The instinct is to negotiate rates or trim prompts. The larger win is usually structural: most requests in a production system are not hard, and they do not need your most capable model.

Segment the traffic first

Before optimising anything, classify a week of real requests by difficulty. In every system we have measured, the distribution is lopsided — a long tail of genuinely hard cases and a fat head of classification, extraction and formatting that a small model handles indistinguishably well.

Route, then verify

• Send the easy head to a smaller, cheaper model.
• Escalate on low confidence, schema failure or explicit refusal.
• Run both models against your eval set and compare — this is exactly what the harness is for.
• Alert on escalation rate, because a drift in traffic mix will change your bill before it changes your metrics.

On the Atlas engagement this pattern removed 41% of inference spend with no measurable quality change on the held-out slice. It took four days. There is no cleverness in it at all, which is rather the point.