Michael Limberger

Not a 2015 Air

Local AI is demanding. Not gaming-PC demanding, but your 2015 MacBook Air is not going to cut it. Check the chip:

sysctl -n machdep.cpu.brand_string

You need Apple Silicon: M1, M2, M3, or M4, any variant. Pro, Max, and Ultra have more memory bandwidth. Base chips work fine for this talk. Intel Macs can technically run the stack, but performance is terrible. On Intel, tonight might be watch-and-learn.

sysctl -n hw.memsize | awk '{print $1/1024/1024/1024 " GB"}'

This is the most important number. Models load entirely into RAM. With 8GB you are limited to small models (about 4B or less). With 16GB you can run medium models one at a time, with a tight context window. With 32GB most models feel comfortable, including the good ones. With 64GB+ you can keep multiple models loaded, use large windows, and stop worrying about headroom. The sweet spot for tonight is 32GB.

On a traditional PC, system memory and graphics memory are separate piles. The model has to fit in the GPU pile. Apple Silicon shares unified memory, so a 64GB Mac Studio can run models that would need a roughly $2,000 NVIDIA card on a PC. Apple accidentally built one of the best consumer AI platforms.

Disk space: Colima plus Docker is about 1GB, the OpenWebUI container is about 2GB, and each model is roughly 3GB to 50GB. Plan for at least 30GB free.

Colima, not Docker Desktop

Two things before OpenWebUI: Colima plus the Docker CLI, and Ollama. If you already have Ollama from a previous session, check ollama --version. You want 0.14.0 or higher.

Colima is a free, open source, command-line Docker runtime for Mac. No GUI, and no license restrictions. Docker on Mac needs three packages: colima (the Linux VM, because Docker needs Linux to run containers), docker (the CLI so you can type commands), and docker-compose (orchestration). Without all three, typing docker says "command not found."

brew install colima docker docker-compose

Docker Compose installs as a standalone binary. Docker does not know about it by default. Register it as a plugin so docker compose (with a space) works:

mkdir -p ~/.docker/cli-plugins
ln -sfn /opt/homebrew/opt/docker-compose/bin/docker-compose \
  ~/.docker/cli-plugins/docker-compose

Without that link you get: "docker: 'compose' is not a docker command."

If you just run colima start with no flags, you get 2 CPUs and 2GB RAM. That is not enough. Start with real resources:

colima start --cpu 4 --memory 8 --disk 60

That is four cores, 8GB RAM, and 60GB disk. Do not give Colima more than about half your total RAM. A 32GB Mac is fine with 8GB for Colima, leaving plenty for Ollama and macOS. On real Apple Silicon (not a nested VM), you can use Apple's native virtualization:

colima start --cpu 4 --memory 8 --disk 60 \
  --vm-type vz --vz-rosetta --mount-type virtiofs

The vz flags use Virtualization.framework instead of QEMU, and it is noticeably faster. That path is only for real hardware. If you are inside Parallels or UTM, stick with the basic command. Settings save, so the next plain colima start remembers them.

Verify with docker ps. You should see an empty list, not an error. "command not found" means you skipped the docker package. "cannot connect to Docker daemon" means Colima is not running.

Auto-start after reboot: brew services start colima. Stop with colima stop. Start again with colima start.

If you do not have Ollama yet: brew install --cask ollama, or download from ollama.com. Open the app so the menu bar icon appears. The API listens on port 11434. It just needs to be running. Verify you are on 0.14.0 or higher. If you are older: brew upgrade --cask ollama.

Checklist before the next page: Apple Silicon, 16GB+ RAM, all three brew packages, the compose plugin linked, Colima started with real resources, docker ps works, brew services start colima if you want auto-start, Ollama 0.14.0+, and about 30GB free. Next: OpenWebUI itself.