Course resource
Temperature and Sampling Guide
What the randomness settings do, and what to set them to.
You will meet these in API calls, automation platforms (the OpenAI module in Make or n8n), and custom-assistant builders. Most consumer chat interfaces do not expose them.
What temperature actually is
The model predicts a probability distribution over the next token. Temperature reshapes that distribution before one is picked.
- Low (0–0.3) — almost always takes the most likely token. Output is consistent and predictable. Run the same prompt twice, get near-identical results.
- Medium (0.5–0.8) — sometimes takes a less likely token. Natural-sounding, varied.
- High (1.0+) — regularly takes unlikely tokens. Surprising, sometimes incoherent.
The crucial misunderstanding: low temperature does not mean correct. It means consistent. A model that is confidently wrong will be confidently, consistently wrong at temperature 0. Temperature controls variety, not accuracy.
What to set it to
| Task | Temperature | Why |
|---|---|---|
| Extracting data from text | 0–0.2 | You want the same answer every time |
| Classification, tagging, routing | 0–0.2 | Consistency is the whole point |
| Code | 0–0.3 | One right answer, mostly |
| Factual Q&A | 0.2–0.4 | Variety adds nothing but risk |
| Summarising | 0.3–0.5 | Slight variation reads better |
| Professional writing, email | 0.5–0.7 | Natural without going strange |
| Marketing copy | 0.7–0.9 | You want options |
| Brainstorming, naming | 0.9–1.2 | You will discard most of it anyway |
| Fiction, poetry | 0.8–1.2 | Surprise is the product |
If in doubt, 0.7. It is the default for a reason.
In automations, go lower than feels right
This is the single most common mistake in production workflows.
A classification step at temperature 0.8 will categorise the same message differently on different runs. You will debug it for an hour before realising the workflow is fine and the temperature is wrong.
Rule: any step whose output another step depends on runs at 0–0.3.
Reserve higher temperatures for the final, human-facing generation step.
Top-p, briefly
Some tools expose top_p (nucleus sampling) alongside temperature. It restricts the choice to the smallest set of tokens whose probabilities sum to p.
top_p: 1.0— no restriction (default)top_p: 0.9— ignore the unlikeliest tailtop_p: 0.5— only the most likely candidates
Change one or the other, not both. They interact in ways that are hard to reason about. Most people should leave top_p at 1.0 and adjust temperature.
Other settings you may meet
| Setting | What it does | When to touch it |
|---|---|---|
max_tokens |
Caps output length | Set it in automations to control cost. Too low truncates mid-sentence. |
frequency_penalty |
Discourages repeating tokens | Raise slightly if output loops |
presence_penalty |
Encourages new topics | Raise slightly for brainstorming |
seed |
Makes output reproducible | Testing and debugging — invaluable for comparing prompts fairly |
stop |
Ends generation at a string | Structured output, or stopping runaway lists |
The experiment
Twenty minutes, and it will make the table above mean something.
Take one prompt. Run it five times at each of 0, 0.7 and 1.2.
| Temperature | How similar were the five? | Quality | Anything unusable? |
|---|---|---|---|
| 0 | |||
| 0.7 | |||
| 1.2 |
Then run it three times at 0 with different seed values, if your tool supports them. Watch what changes and what does not.
Diagnosing by symptom
| Symptom | Likely cause | Fix |
|---|---|---|
| Same task, different answers each run | Temperature too high for the job | Drop to 0–0.2 |
| Output is stiff and repetitive | Temperature too low for writing | Raise to 0.6–0.8 |
| Output loops or repeats phrases | Low temperature, or long context | Small frequency penalty, or shorten input |
| Occasionally goes off the rails | Temperature above 1.0 | Lower it |
| Truncates mid-sentence | max_tokens too low |
Raise it |
| Cannot reproduce a good result | No seed set | Set a seed while testing |
The thing to remember
Temperature is not a quality dial. Turning it down does not make the model smarter and turning it up does not make it more creative in any useful sense — it makes it less predictable. Accuracy comes from context, constraints and verification. Temperature only decides how much the output varies between runs.