Context Window
What is it?
A context window is the maximum amount of text (measured in tokens) a language model can consider at once — its short-term memory for a request.
Explain like I'm 5
Why was it created?
Models can't process unlimited text at once. The context window defines how much they can take in for a single response.
Where is it used?
- Fitting prompts and documents
- Long conversations
- Summarization limits
- Deciding when to use RAG
Why should developers care?
It limits how much you can feed a model, shaping prompt design, document handling, and the need for techniques like RAG.
How does it work?
The model processes a fixed maximum number of tokens — your prompt plus its reply must fit within that limit. Content beyond the window must be dropped, summarized, or retrieved selectively.
Real-world example
To ask about a 500-page manual that exceeds the window, you retrieve only the relevant sections (via RAG) instead of pasting the whole thing.
Common use cases
- Managing long inputs
- Multi-turn conversations
- Choosing RAG over full-text
- Budgeting prompt size
Advantages
- Defines clear input limits
- Larger windows handle more context
- Helps reason about model capacity
Disadvantages
- Hard limit on input size
- Larger windows cost more and can be slower
- Important info can fall outside it
- Models may use the middle of long contexts less effectively
When should you use it?
Whenever you decide how much text to send a model in one request.
When should you avoid it?
Not avoidable — it's a model property to design around.
Alternatives
Related terms
Interview questions
Beginner
- What is a context window?
- What is it measured in?
Intermediate
- Why does the prompt and reply share the window?
- How does RAG help with window limits?
Senior
- What are the trade-offs of very large context windows?
- How do you handle content that exceeds the window?
Common misconceptions
- "Bigger context window means the model remembers forever" — it's per-request short-term memory, not permanent storage.
- "More context always gives better answers" — relevance matters more than volume, and huge contexts can dilute focus.
Fun facts
- The window counts both your input and the model's output together.
- Context windows have grown dramatically, from a few thousand tokens to hundreds of thousands or more.
Timeline
- 2020s — Context windows expand rapidly across model generations
Learning resources
Quick summary
A context window is the maximum tokens a model can consider at once — its working memory, shared between your prompt and its reply.
Cheat sheet
- Max tokens per request
- Model's short-term memory
- Prompt + reply must fit
- Use RAG when content exceeds it