01Deep dive
Meaning and practical implications
An application can send much more than the user’s visible question. Inspect the assembled request to find repeated instructions, redundant documents, or growing conversation history.
Prompt tokens are the tokens in the input supplied to a language model, including applicable instructions, conversation history, retrieved material, and tool results.
An application can send much more than the user’s visible question. Inspect the assembled request to find repeated instructions, redundant documents, or growing conversation history.
Paste a week of traffic. Get the number that books the audit.
See what you're overpaying →