What is tokens?
Tokens are units a model uses to represent input or output. Language-model APIs often price input and output tokens separately. Token counts vary by tokenizer, language, content, and endpoint.
o10Reviewed 2026-09-06
AI tokens: input, output and the cost of context
Tokens are units a model uses to represent input or output. Language-model APIs often price input and output tokens separately. Token counts vary by tokenizer, language, content, and endpoint.
SummaryKey takeaways
What you need to know
Start with the core questions, then examine the examples and tradeoffs below.
01Deep dive
Count what the endpoint bills
A prompt can include instructions, conversation history, retrieved passages, tool descriptions, and other context. Output usage may include more than the text shown to the user, depending on the endpoint. Consult that endpoint's usage fields and billing documentation.
Do not treat a word-to-token estimate as an invoice calculation. Measure real usage for representative requests, especially for code, multilingual content, or structured data.
02Deep dive
Calculate input and output separately
Multiply input tokens by the input rate and output tokens by the output rate, using the price unit advertised by the provider. Add other billed usage. A blended rate is valid only for the particular input/output mix used to calculate it.
Illustrative example: 1,000 input tokens at $2 per million cost $0.002. Another 1,000 output tokens at $6 per million cost $0.006. The total is $0.008 before other charges; doubling the output changes the cost even though the prompt is identical.
03Deep dive
Reduce context without losing required evidence
Remove redundant instructions and unrelated retrieval results, and set output limits that still allow the task to finish. Check whether the chosen endpoint supports caching or batch processing and whether your traffic meets its conditions.
Shorter is not automatically better. If removing a source causes incorrect answers or additional retries, the application may spend more per accepted result. Evaluate changes using task outcomes as well as token totals.
SourceMethodology
Cost examples are illustrative. Measure quality and total costs on your own tasks, and check current endpoint terms before implementation.
FAQFrequently asked questions
Common questions
o10Set the envelope. o10 holds it.
See what you're overpaying.
Paste a week of traffic. Get the number that books the audit.
See what you're overpaying →verified savings methodology · State of Inference Spend 2026