Context window
context length · context limit
Kurz gesagt
The context window is the maximum number of tokens a model can process in a single request. It holds the system prompt, tool definitions, conversation history, any attached documents and the generated output. Anything outside the window is invisible to the model.
The context window is a hard ceiling, and it is a shared budget. Everything competes for the same space: system instructions, tool schemas, the conversation so far, retrieved passages, attached files, and the response being generated.
That last item catches people. A very long input can leave insufficient room for a long answer, producing output that stops mid-sentence for no obvious reason.
Long contexts also degrade unevenly. Research on long-context behaviour found a U-shaped accuracy curve: facts at the beginning and end of a long input are recovered far more reliably than facts in the middle. Practical rules follow directly — instructions first, the immediate question last, and no padding.
Cost tracks the same budget. Most chat APIs re-send the entire conversation on every turn, so a long thread grows expensive faster than the number of messages suggests. Prompt caching reduces the price of an unchanged prefix, but only when that prefix is byte-identical between requests.
The usual remedy for hitting the limit is not a bigger window but retrieval: fetch the few passages that bear on the question and send those.
Häufige Fragen
- Does a larger context window mean better answers?
- Not automatically. Models retrieve information placed near the start and end of a long input more reliably than material in the middle, so relevance matters more than volume.
- What happens when a conversation exceeds the window?
- The application must drop, summarise or externalise older turns. The model cannot see anything outside the window and will not warn you that something was removed.