Michael Limberger

Context Management

Context management

A good chatbot is not only the model. It is also what the model is allowed to see. Get context management right and the character stays consistent through a long chat. Get it wrong and the persona dilutes, repeats itself, or invents both sides of the conversation.

Stop sequences: why the chatbot keeps talking

Without stop sequences, the model often generates both sides of the chat. You send a user line, and it continues into another user line of its own:

AI: "Your favorite book? Well, that would have to be Moby Dick..."
        User: "Oh yeah, I love that one too!"
        AI: "Of course you do. Everyone with taste does..."

Stop sequences cut generation before that runaway turn-taking starts.

The essential stop sequences

At minimum, add:

"User:"
        "\nUser:"

Those catch the model before it writes your next message. If the character has a fixed name (for example Mark), add variants of that name too, plus the template placeholders if you use them:

"Mark:"
        "\nMark:"
        "{{user}}:"
        "\n{{user}}:"

Setting stop sequences across platforms

In an Ollama Modelfile:

PARAMETER stop "User:"
        PARAMETER stop "\nUser:"

You can add multiple PARAMETER stop lines. Ollama reads all of them.

In an OpenWebUI API call:

{
          "model": "magidonia",
          "messages": [...],
          "stop": ["User:", "\nUser:"]
        }

In the OpenWebUI UI: Advanced, Stop Sequences, then add User: and \nUser: on separate lines.

Advanced > Stop Sequences > Add "User:" and "\nUser:"

Enter each on a separate line.

Multiple characters: expand the stop list

For a group scene, stop on every speaker name so one voice does not turn into a monologue:

PARAMETER stop "User:"
        PARAMETER stop "\nUser:"
        PARAMETER stop "Alice:"
        PARAMETER stop "\nAlice:"
        PARAMETER stop "Bob:"
        PARAMETER stop "\nBob:"

The chat template: Mistral V7-Tekken

Magidonia uses the Mistral V7-Tekken chat format under the hood. Ollama usually applies it automatically, so you do not hand-roll special prompt wrappers for basic chat. The roles stay familiar:

  system: Your character description
          user: User's message
          assistant: Character's response
        
        Ollama does the rest.

The context window problem

Your character card might be a few hundred to a thousand tokens. The growing chat can be thousands more. Everything shares one window. When the window fills, older turns fall out or get compressed, and the persona at the front of the prompt gets diluted by everything that came after.

The paradox of large contexts

Bigger context does not automatically mean better character consistency. On a high-memory Mac you could run a 32K window with Magidonia Q8_0. Technically that works. In practice, very large windows often make the model forget core traits while obsessing over recent filler. The earlier recommendation of about 8K to 16K still holds for character work: enough depth for a real conversation, not so much that the card drowns in noise.

Context management strategies

When chats get long, use a few deliberate tactics.

Context pruning: drop logistics and dead ends that do not advance the relationship or the scene. Early turns often matter; middle turns get repetitive; late filler is expensive to carry forever.

Summarization: write a short "so far" note and keep that instead of fifty redundant messages:

"So far: Alice learned I'm from Vermont. We discussed the climate crisis for 10
         exchanges. She asked about my childhood. I shared the barn story. Current mood:
         reflective, slightly nostalgic, engaged."

Character refresh: reinject a short trait reminder near the end of the window so attention sees it again:

[Character message from 5 exchanges ago...]
        [User response...]
        [Character message...]
        [User response...]
        ← Inject reminder here: "Mark Twain: sardonic, skeptical of authority, loves storytelling.
          Always speaks plainly. Never preachy."
        [User asks a new question...]
        [Model generates response, now with the reminder fresh in its attention]

Why depth matters: recency bias

Language models pay more attention to tokens near the end of the context than to tokens near the beginning. That is how attention tends to behave, not a Magidonia quirk. A trait list sitting only at the front will fade as the chat grows. Refreshing the persona later in the window fights that drift.

Implementing character refresh

In newer SillyTavern builds you can use Character's Note so the client injects a reminder automatically. In OpenWebUI or a plain log, do it by hand:

After 20-30 messages, pause the conversation.
        Edit the chat log. Find a natural spot (end of a character message).
        Add a bracketed reminder:
        
        [Mark is a storyteller, skeptical of government, loves technology and river life.
         Speaks plainly. Often folksy. Opinionated but friendly, not preachy.]
        
        Continue the conversation. The model sees the reminder.

Signs of drift

You will notice drift when replies go generic, speech patterns change, the character suddenly uses vocabulary they never had, they agree with everything, or they forget facts established earlier. Note roughly which message count it starts. Early drift usually means the window is too small or the card is too bloated.

Fixes for drift

Check context size and keep it in the character-friendly range.
Add a character refresh note.
Summarize and prune older turns.
Regenerate the last reply (a bad sample happens).
Edit an out-of-character line so the history teaches the right tone.
Reset with a summary when the thread is exhausted.

Managing memory explicitly

For plot, world-state, and relationship facts, do not rely on the model alone. Keep a short KNOWN_FACTS note outside the chat and paste updates when needed.

KNOWN_FACTS:
        - Alice knows I'm from Vermont
        - We talked climate for about ten exchanges
        - Shared the barn childhood story
        - Current mood: reflective, slightly nostalgic

The nuclear option: reset with summary

Sometimes a hundred-message thread is repetitive or badly drifted. Starting over is not failure. Write a three to five sentence summary of what mattered, open a new chat, paste the summary near the top, and continue with a fresh window.

"Alice and I spent the morning discussing philosophy. She's skeptical of my atheism but
         respects my reasoning. We found common ground on the nature of meaning. By the end,
         she was laughing at my dry jokes. Mood: warm, intellectual, slightly flirtatious."

Keep this nearby

Context management checklist:

☐ Stop sequences set ("User:", "\nUser:")
        ☐ Context window at 8K-16K (not higher "just because")
        ☐ Character refresh after roughly 30+ messages
        ☐ Summarization every 50 messages or at natural breakpoints
        ☐ Manual edit of out-of-character responses
        ☐ KNOWN_FACTS maintained externally for plot/world details
        ☐ Topic segmentation (don't force everything into one chat)
        ☐ Reset with summary when drift is terminal

Next we put this into practice with full character builds, starting with Mark Twain.