Michael Limberger

The words, said out loud

AI has a lot of jargon. You are welcome here if you can open a terminal and you are new to this craft. We name each word the first time it shows up, then we use it.

The brain, and the act of running it

A large language model (LLM) is the AI brain: a neural network trained on massive amounts of text that predicts what words come next. When you "chat," you are asking it to complete your text in a helpful way. GPT-4, Claude, Llama, Mistral are all LLMs.

Inference is the act of running that model to generate text. When the AI "thinks" and produces a response, that's inference. It is computationally expensive, which is why cloud services cost money and local use wants decent hardware.

On your machine, or on theirs

Cloud means the AI runs on someone else's servers: OpenAI, Anthropic, and the rest. You send prompts over the internet. They see everything.

Local means the AI runs on your computer. Nothing leaves the machine. Private, free after setup, and it needs decent hardware. Tonight we go 100% local.

The dials

Parameters (also called samplers) are settings that control how the model generates text. Think of them as personality dials, not as the model's size.

Temperature is how creative or random versus focused. Top P is how much probability mass to consider. Top K is the maximum number of next-word candidates. Min P is a floor that drops garbage tokens. Repeat penalty is how hard the model avoids repeating itself.

We'll set the ones that work for roleplay later. A token is the unit an LLM actually works with: not quite a word, not quite a letter. Roughly one token is about 4 characters, or 0.75 words. "Hello, how are you today?" is about 7 tokens.

Why it matters: models have token limits. A 32,000 token context is about 24,000 words of history.

How much it can see at once

The context window is how much the model can "remember" at once. Everything inside it influences the response. Anything outside is forgotten.

8K context is about 6,000 words. 32K is about 24,000. 128K is about 96,000. Bigger is not always better. We'll say why later.

A system prompt is the instruction given before the conversation starts. It sets how the AI should behave. Usually hidden from the user. Example: "You are a helpful assistant who speaks like a pirate."

A character card is a structured description: personality, background, speech patterns, example dialogues. The blueprint for your companion.

Two kinds of models

An instruct model is trained to follow orders. "Do X." Good for tasks. Mistral Instruct and Llama Instruct live here.

A roleplay model is trained on collaborative fiction. "Continue this scene." RPMax, Fimbulvetr, Noromaid live here. Different training, different behavior. We want roleplay models.

Fine-tuning means taking a base model and training it further on specific data. RPMax is Mistral that has been fine-tuned on roleplay conversations.

Compression, and the file on disk

Quantization is compressing a model so it uses less memory. Tradeoff: size versus quality. Q8 is highest quality, largest. Q6 is the sweet spot. Q4 is smaller and still good. Q2 is smallest and you feel it.

We'll use Q6_K_L for the best balance. GGUF is the file format for those quantized models. When you download a model for local use, it is usually a .gguf file. Pronounced "goof" by some, "G-G-U-F" by others. Nobody really knows.

Wrappers, stops, and when the voice slips

A prompt template (or chat template) is the formatting wrapper around your messages. Different models expect different shapes.

ChatML looks like <|im_start|>user then the text then <|im_end|>. Llama uses [INST] and [/INST]. Alpaca uses ### Instruction: and ### Response:. Use the wrong template and the model gets confused.

Stop sequences are tokens that tell the model "stop generating here." They keep it from writing the other person's lines or wandering off.

Character drift is when the voice slowly goes generic over a long chat. Starts strong, becomes beige. One of the main problems we're solving.

Breaking character is sudden: the AI stops being the person and becomes a generic assistant. "As an AI language model, I..." That's a break.

OOC means out of character: a note between you and the model that is not part of the scene. Usually wrapped in brackets: [Can you make responses shorter?]

The hobbyists who figured this out

These techniques did not come from corporate labs. They come from hobbyist communities that spent years making AI characters that don't break. Let's name them.

SillyTavern is a free, open-source chat interface for AI models. Think of it as a power-user alternative to ChatGPT's window, built for character roleplay. It started as a fork of TavernAI and became the go-to tool for serious hobbyists. It runs locally and talks to backends: local models via Ollama, or cloud APIs.

SillyTavern users developed many of the card formats we'll use. When you see PList or Ali:Chat, those came from this community.

Features we will not cover tonight: lorebooks (keyword-triggered world info), author's notes (mid-context injections), group chats, regex scripts that post-process replies, advanced memory systems. We're taking their formatting wisdom and applying it to simpler setups.

Chub.ai is a website where people share character cards. A library of pre-made personalities you can download. Thousands of characters, ratings, reviews, tips. Best practices we teach come from patterns that showed up there across thousands of users.

What dominates Chub.ai that we're skipping: anime and manga characters, romantic companions, fandom characters, and a significant portion of NSFW. We're using their techniques for different purposes.

A TavernAI card stores everything in a PNG image file. Name, personality, examples, all embedded invisibly in the image metadata. Share one image, load a complete character. OpenWebUI can import these with basic support. SillyTavern uses them natively. We won't create them tonight. You might meet them.

Same methods, different people

The roleplay community skews toward anime, romance, fantasy, and fanfic. The top 10 most-used characters are almost all from anime: Jujutsu Kaisen, Genshin Impact, and so on.

Tonight we apply the same techniques to historical figures (Mark Twain, Marcus Aurelius), practical companions (study buddies, journaling partners), supportive but non-romantic characters, and original personalities for a job. Same methods. Different application.

World info, and a note at a depth

A lorebook (world info) auto-injects text based on keywords. You set an entry for "Rivendell" with lore about the elven city. Whenever "Rivendell" appears, that lore is added to context. The AI "knows" the details without stuffing the system prompt. Useful for big worlds. OpenWebUI does not have this. If you need world info, put it in the system prompt.

An author's note (or character's note) injects text at a specific depth: a certain number of messages back from the current one. Models pay more attention to text near the end of context. That is recency bias. Information at the top of a long chat gets diluted. Author's notes put reminders closer to where the model is generating.

Example: [Remember: Luna asks questions, doesn't give advice] at depth 4 keeps the character on track. We'll call this character refresh when we get to context management.

Now that you speak the language, let's look at the hardware.