Engineering · 2 min read
Model overkill — paying premium prices for commodity work
The most common AI cost leak isn't a bug. It's a premium model quietly answering questions a model a tenth of the price would handle just as well.
By Akhil Anand · September 3, 2026
There's a cost leak in most AI products that never shows up as an error, an alert, or an anomaly. It's a top-tier reasoning model being used to classify sentiment, extract a field, or reformat some text — work a small, cheap model does perfectly well. Everything works. You just overpay for it, on every request, forever.
The silent premium
Teams reach for the best model during prototyping — reasonably, because it removes a variable while you're figuring out if the feature works at all. Then the feature ships, traffic grows, and nobody revisits the model choice. The premium that was harmless at 100 requests a day is a line item at 100,000.
The gap is often 10–30× per token between a frontier model and a capable small one. For a task the small model handles at equal quality, that entire multiple is pure waste.
Nothing broke. The feature works. You're just paying reasoning-model prices for commodity inference — and the invoice is the only place it shows.
Spotting it
Overkill has a signature you can look for:
- Simple task, expensive model. Short inputs, deterministic outputs, a frontier model on the call.
- No quality gap. A/B the cheap model and the expensive one — if users can't tell, you're overpaying.
- High volume. The leak scales with request count, so your highest-traffic endpoints are where the money is.
Route by task, not by default
The fix isn't "always use the cheap model" — some work genuinely needs the frontier. The fix is to match the model to the task instead of defaulting everything to the most capable option. Cheap model for extraction and classification; premium model for genuine reasoning and generation.
You can't route what you haven't measured, though. Model overkill is invisible until you break cost down per feature and per model and see the premium model sitting on commodity work. Measure first, then move the easy traffic down a tier — it's usually the fastest real dollar you'll save.