Michael Limberger

Job description versus work style

Advanced Parameters control how the model generates text, not what it knows. The system prompt is the job description. Parameters are work style. Same person, same job: play it safe or take risks? Ramble or keep it tight?

You can set them in two places. In Workspace under Models, Create or Edit, Advanced Parameters applies to every conversation with that custom model. In a live chat, the gear near the model dropdown applies to this conversation only. Use the chat gear for experimenting. Bake the winners into a custom model when you like them.

Temperature is the big one. It controls randomness in word choice. Low (0.1 to 0.3) sticks to the highest-probability word: focused, and the same question twice tends to produce nearly the same answer. Medium (0.5 to 0.7) is a balance and a good default. High (0.8 to 1.5) is more creative and sometimes weird. Above 1.5 often turns incoherent. Defaults are usually 0.7 or 0.8. For code try 0.2 or 0.3. For facts try 0.3 to 0.5. For creative writing try 0.8 to 1.0. For brainstorming try 1.0 to 1.2. For roleplay try 0.8 to 1.0. Temperature 0 means always pick the most likely next word. Temperature 2 means surprise me.

Top P (nucleus sampling) cuts the candidate list. It adds probabilities until they reach P. 0.1 is very focused and can get repetitive. 0.9 is a wide pool of reasonable words. 1.0 is everything, which effectively turns Top P off. Default is usually 0.9. Temperature makes unlikely words more likely. Top P removes them. Think volume knob versus filter. Rule of thumb: adjust temperature or Top P, not both at once.

Top K is simpler: only the K most probable next words. Common values are 10, 40 (a common default), and 100. Top P adapts to the distribution. Top K is a fixed count. Most people leave Top K at 40.

Repeat penalty: 1.0 is off, 1.1 is mild (a good start and the usual default), 1.2 is moderate, and 1.5+ often hurts coherence. Repeat Last N is how far back the penalty looks, typically 64 tokens.

Context length (num_ctx) is short-term memory. 2048 is about 1,500 words. 4096 is about 3,000. 8192 is about 6,000. 32768 is about 24,000. Exceed it and the oldest messages are gone. Not forgotten: they no longer exist in working memory. Larger context uses more RAM. Doubling context roughly doubles the KV cache. A model that needs 8GB at 4K might need 16GB+ at 32K. Do not crank it without checking memory. In OpenWebUI you can set context per custom model, or bake it into an Ollama Modelfile. The UI option is easier for most people.

Max tokens (num_predict) limits a single response, not the whole conversation. 256 is short, 1024 is medium, 4096 is long, and -1 means no limit. Seed makes output reproducible for testing. Leave it 0 or blank for variety. Stop sequences halt generation when the model is about to emit them. Min P is a relative floor; try 0.05 to 0.1. Mirostat auto-adjusts randomness. When Mirostat is on, Top P and Top K are typically ignored.

Starting points worth copying. Coding: temperature 0.2, Top P 0.9, repeat penalty 1.1. Conversation: 0.7 / 0.9 / 1.1. Creative writing: 0.9 / 0.95 / 1.15. Factual Q&A: 0.3 / 0.85 / 1.0. Next: vision models.