Temperature
What is it?
Temperature is a setting that controls how random a language model's output is — low temperature makes it focused and predictable, high temperature makes it varied and creative.
Explain like I'm 5
Why was it created?
A model produces a probability for each possible next token. Temperature was introduced as a simple knob to trade off between reliable, repeatable answers and diverse, creative ones.
Where is it used?
- Chat and completion APIs
- Creative writing tools
- Deterministic extraction tasks
- Sampling configuration
Why should developers care?
It's one of the most common settings you'll tune when building with LLMs, and it directly affects reliability, so understanding it saves a lot of trial and error.
How does it work?
Before picking the next token, the model has a probability for each option. Temperature rescales those probabilities: low values sharpen them (the top choice dominates), high values flatten them (less-likely tokens get a real chance). Temperature 0 makes it (nearly) deterministic.
Real-world example
For extracting a date from text you set temperature near 0 for consistency; for brainstorming taglines you raise it to get varied ideas.
Common use cases
- Deterministic outputs (temp ~0)
- Creative or varied output (higher temp)
- Reducing repetition
- A/B testing prompt behavior
Advantages
- Simple, single knob
- Trades reliability for creativity on demand
- Supported by virtually every model API
- Cheap to experiment with
Disadvantages
- High values increase errors and drift
- Interacts with other sampling settings (top-p)
- Not a quality control by itself
- Same value behaves differently across models
When should you use it?
Lower it for factual, structured, or repeatable tasks; raise it for brainstorming and creative variety.
When should you avoid it?
Don't rely on temperature alone to fix quality or accuracy — grounding and prompting matter more.
Alternatives
Related terms
Interview questions
Beginner
- What does raising temperature do to model output?
- What temperature would you use for consistent extraction?
Intermediate
- How does temperature reshape the token probability distribution?
- How do temperature and top-p interact?
Senior
- Why can temperature 0 still produce different outputs across runs or hardware?
- When would you prefer top-p over temperature?
Common misconceptions
- "Higher temperature is smarter" — it's just more random, often less accurate.
- "Temperature 0 is fully deterministic everywhere" — implementation and hardware details can still cause variation.
Fun facts
- The name comes from the physics of the softmax function used to sample tokens.
- Many production extraction pipelines run at temperature 0.
Timeline
- 2010s — Softmax temperature used across neural network sampling
- 2020s — Becomes a standard knob in LLM APIs
Learning resources
Quick summary
Temperature is a knob for output randomness: low values give focused, repeatable answers; high values give varied, creative ones.
Cheat sheet
- Creativity/randomness dial
- Low = focused, high = varied
- ~0 for extraction, higher for ideas
- Pairs with top-p sampling