Serving
Context window
What is context window?
The context window is the maximum number of tokens a model can attend to at once, covering the prompt and the generated output together. Exceeding it forces truncation or a chunking strategy, both of which lose information.
Long windows remove architectural complexity — a whole document in one pass needs no retrieval layer and no chunk-merging logic, and those layers are where subtle errors accumulate.
They are not free. Attention memory grows with sequence length, so a long window used to its limit costs considerably more per request than the same model used briefly.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.