LLM settings (temperature, top-p, top-k, max tokens, frequency and presence penalties) are parameters available through APIs and playgrounds like OpenAI Playground or Google AI Studio that control how a large language model picks its next words. This lesson explains what each setting does and gives starter recipes for extraction, summaries, planning, brainstorming, and writing.
What you will be able to do
- Set temperature, top-p, and max tokens to match a task instead of guessing
- Pick starter recipe values (temperature and top-p) for extraction, summaries, planning, brainstorming, and writing tasks
- Diagnose and fix common output problems: topic drift, cut-off answers, repeated words, and broken JSON
- Use a seed parameter to get repeatable results for tests or demos
- Reduce repetition in output with frequency and presence penalties
Before you start
- Access to an LLM API or playground such as OpenAI Playground or Google AI Studio, since these settings are not available in standard ChatGPT or Claude chat interfaces
- An account with the AI provider (free trials are available)
- Basic familiarity with prompts and how large language models generate text, covered in the earlier lesson on how LLMs work
Reference
| Setting | Range | What it does | When to use |
|---|---|---|---|
| Temperature | 0.0 to 2.0 (above 1.0 rarely useful) | Controls randomness in token selection; low is predictable, high is creative | 0.0-0.3 for extraction, code, math; 0.4-0.7 for summaries and writing; 0.8-1.0 for brainstorming |
| Top-p (nucleus sampling) | 0.0 to 1.0, commonly around 0.9 | Picks from the smallest group of tokens whose probability adds up to p | Leave at 0.9 for most tasks; lower to 0.8 for careful work, raise to 0.95 for brainstorming |
| Top-k | integer, e.g. 40 | Picks only from the top k most likely tokens by rank | Rarely touched; some providers use this instead of top-p |
| Max tokens (output length) | numeric limit | Sets the maximum response length; the answer cuts off at the limit | Set high enough for the expected output, or add a length target in the prompt |
| Stop markers | custom strings such as , STOP, END | Tells the model when to stop generating | Use for strict formats like JSON to cut off extra text |
| Seed | any number you choose | Makes output repeatable with the same prompt and settings | Use when exact repeatability matters, such as tests or demos |
| Frequency penalty | positive value, try 0.3 to 0.5 | Discourages words that already appeared multiple times, more each time | Raise if the output repeats the same word many times |
| Presence penalty | positive value, try 0.3 to 0.5 | Discourages any word that appeared even once, encouraging new topics | Raise if the model keeps circling the same topics |
Common errors and fixes
| Problem | Fix |
|---|---|
| Answer drifts off topic | Lower temperature a little, add a clear rule restating the goal, and add a refusal rule for low confidence |
| Answer is too short or cuts off | Add a word or token target in the prompt, increase max output length, and remove extra context to leave room |
| Answer is boring or stiff | Raise temperature a little, raise top-p a little, and add a style note or example |
| Answer repeats words or lines | Add a small frequency penalty and ask the model to avoid repeating phrases |
| JSON breaks your parser | Lower temperature, add a strict schema and example, tell the model to produce only JSON with no narrative text, and add a stop marker if supported |
Read the full walkthrough
The complete lesson, with screenshots and any downloads, is published on Substack as part of Prompt Engineering for AI Automation.
More in this section
- Lesson 1: LLMs and Prompts in Generative AI
- Lesson 3: Zero-Shot, One-Shot, and Few-Shot Prompting Explained
Continue the course
Browse all lessons in the Prompt Engineering for AI Automation course, or subscribe to the GenAI Unplugged newsletter to get new lessons in your inbox.
