Engineering · 1 min read
Capturing token usage across OpenAI, Anthropic, and Gemini
Every provider reports usage differently, and streaming hides it in a different place again. Here's why cost instrumentation is annoying — and where to do it once.
By Akhil Anand · August 31, 2026
You'd think reading how many tokens a request used would be simple. It is — until you support more than one provider. Then it becomes a small zoo of field names, response shapes, and streaming quirks that quietly breaks your cost tracking.
The field-name zoo
The same concept — input tokens — has a different name in every provider's response:
OpenAI: usage.prompt_tokens / usage.completion_tokens
Anthropic: usage.input_tokens / usage.output_tokens
Gemini: usageMetadata.promptTokenCount / candidatesTokenCount
Miss a variant and that provider's spend silently reads as zero. Your dashboard looks fine; it's just wrong. And "wrong but confident" is the worst state a cost tool can be in.
Streaming makes it worse
Non-streaming responses put usage in the body. Streaming responses (SSE) often don't include usage until the final chunk — and sometimes only if you asked for it. If your instrumentation reads the first chunk and moves on, every streamed call counts as free.
The most expensive requests — long agent turns, big generations — are the ones most likely to stream. Miss usage on streams and you under-count exactly where it hurts.
Do it once, at the fetch layer
The mistake is instrumenting per-call, per-provider, scattered through your code. Every new endpoint is a new place to forget. The durable pattern is to capture usage once, at the network boundary — wrap fetch, detect the provider from the host, and parse usage (JSON and SSE) in one place.
That's exactly how the AtlasBurn SDK works: it patches fetch, recognizes calls to OpenAI, Anthropic, and Gemini/Vertex, and extracts token usage across all the field-name variants and streaming shapes — so adding a new AI call to your app adds nothing to your cost-tracking burden. Instrument the boundary once, and every future call is covered by default.