Input tokens and output tokens

Input tokens are everything sent to the model on a request; output tokens are what it writes back, and in agent work the input side dominates the bill by an order of magnitude or more.

The asymmetry is the whole point. An answer is a few hundred to a few thousand tokens. The context that produced it is routinely tens of thousands, and unlike the answer it is re-sent on the next turn, and the one after that.

This is why advice to make the model answer more briefly has almost no effect on cost, and why advice about what to send has a large one.

Input is also priced differently depending on whether it is fresh or cached. Cached input is billed at a fraction of the fresh rate, which means two sessions with identical raw token counts can cost very different amounts. Any honest comparison of token usage has to say which of the two it is measuring.

All terms