Michael Limberger

Two pieces that keep a long night coherent

Stop sequences prevent the model from generating too much. Context management keeps long conversations coherent. Stop sequences are tokens that tell the model "stop generating here." When the model produces one of them, generation halts immediately.

Without them, User asks how Luna is, Luna answers, then the model writes User's next line and Luna's next line and keeps going. It played both parts. With stop sequences, generation stops when it tries to write User:. Much better.

The minimum stops

For a companion, you need at least: User:, User:, {{user}}:, and {{user}}:. That covers variations in how the user's name might appear.

Show me. Ollama Modelfile:

PARAMETER stop "User:"
PARAMETER stop "
User:"

Ollama API request:

{
  "model": "rpmax",
  "prompt": "...",
  "stop": ["User:", "
User:"]
}

OpenWebUI: Settings, Model, Advanced, Stop Sequences. SillyTavern handles this automatically in Chat and Instruct modes. Bolt AI: Preferences, Generation, Stop Sequences.

Multiple AI characters (a group chat, or a multi-persona forum) get all names: User, Luna, Alex, Marcus. Generate one character's response at a time. Call the model separately for each. The stops prevent bleed-over.

The wrapper can be a stop too

Different models expect different formatting. RPMax uses Mistral's format:

<s>[INST] {system} [/INST]
{assistant}</s>
[INST] {user} [/INST]

The tokens in this template can also be stop sequences: </s> and [INST]. Most interfaces handle this automatically. If you're calling the API directly, get the template right or the model gets confused.

The window fills up

Your model has a context window: the maximum text it can "see" at once. For RPMax, that's 32K tokens (about 24,000 words). Sounds like a lot. Here's what fills it: system prompt about 500 tokens, character card about 800, chat history the rest. A 50-message conversation easily hits 10 to 15K tokens.

When the window is full, old messages get truncated. The model forgets what happened earlier. That causes character drift.

You'd think bigger context means better memory. Research shows character consistency drops with very large contexts. The model has more to pay attention to. Important traits get diluted. The card at the top matters less when there's 30K tokens between it and the current message. Sweet spot: 8K to 16K tokens. Beyond that, more "memory," less consistent voice.

Prune, summarize, refresh

Context pruning: remove irrelevant messages. Keeping every "How are you?" / "I'm fine" wastes tokens. Prune routine exchanges.

Summarization: every 30 to 50 messages, write a short recap. "Previous conversation: User discussed work stress, mentioned a conflict with their boss named Sarah, expressed frustration about lack of recognition. Luna offered support, asked clarifying questions." Put that summary in context. Remove the old messages.

Character refresh: inject reminders mid-context. SillyTavern calls this author's note or character's note. Place at depth 4 (4 messages from the end): [Remember: Luna is warm, curious, asks questions, keeps it brief]. That refreshes who it's playing.

Why depth 4, not the top of the file

AI models pay more attention to text near the end of context than text at the beginning. That's recency bias. The attention mechanism weighs recent tokens more heavily. Information at the top of a 10K token context gets diluted. Information near the current message stays sharp.

Your character card sits at the top (system prompt). As the conversation grows, it gets farther from the action. The model "forgets" traits because they're too far back. Injecting traits at depth 4 puts them near the current message. Same information, different position, much stronger effect.

If you read a 50-page document and someone asks a question, you remember the last few pages better than page 3. LLMs work similarly. That's why the SillyTavern community developed author's notes: they discovered that trait positioning matters as much as trait content.

Topic segmentation: start new chat sessions for new topics. Don't have one infinite conversation about everything. Keep conversations focused.

How to actually inject the reminder

SillyTavern has this built in: Character, Advanced, Character's Note. Set depth to 4. Content: [Luna: warm, curious, asks follow-ups, keeps it brief].

Manual method, in the system prompt, add after every N messages: [Reminder: Luna stays in character, maintains her warm but casual voice, asks questions instead of giving advice].

Depth 0 is right before the current message (strongest). Depth 4 is 4 messages back (moderate). Top of context is the first thing in the window (weaker). Character cards are at the top. Refreshers at depth 4 help maintain voice.

When the voice starts to beige

Signs: generic responses ("I understand your feelings..."), changing speech patterns (formal when it should be casual), forgetting established details, breaking the fourth wall ("As an AI..."), becoming a yes-man.

Fixes: check context size. Add character refresh at depth 4. Summarize and prune. Regenerate the drifted response. Edit the response manually to correct course.

Since context is limited, you can keep important facts outside the chat:

KNOWN_FACTS:
- User's name: Alex
- User's job: Software developer
- User mentioned: Recent breakup, stress at work
- User likes: Science fiction, coffee, hiking

Include relevant facts in the system prompt. Update as you learn things. Important details don't get lost to truncation.

Sometimes drift gets too bad. Don't fight it. Reset the chat. Save a summary of important details, start a new conversation, paste the summary as context. Fresh start with memory preserved.

Minimum stops: User: and User:. Add all character names in multi-char scenes. Add chat template tokens if using the raw API. Keep context at 8K to 16K. Refresh at depth 4. Prune or summarize every 30 to 50 messages. Start new chats for new topics. Save facts externally. Don't let bad messages stand. Regenerate or edit immediately.