Michael Limberger
Need me? Email mike@limberger.ca
Terms
Glossary
Plain English for the score labels
Harness
The automatic test runner (skiff-evals). It checks whether the agent finished the tasks and how clean the tool use was.
Harness grade
Letter grade from that automatic score, ranked only against other models in the same setup (local vs cloud).
Vibe
My eye test on the HTML. Does the page look finished, or awkward? Not part of the automatic score.
Vibe grade
Letter grade from that look check (A, B+, B, and so on).
Hygiene
Cleanliness of the run. Starts at 100. Drops for failed tools, retries, and reading the same file again when it did not need to.
Pass / 100%
The checklist said the task outputs were good enough. Many models hit 100%, so this alone does not pick a winner.
Turns
How many back-and-forth steps the agent took with the model before it finished.
Sum
Total wall-clock time for both tasks added together.
Local MLX
Models running on my Mac Studio M2 (AI model server) with Apple's MLX stack.
Ollama Cloud
Vendor-hosted models reached through Ollama on the Mini. Faster here, but not self-hosted.
Complexity
How ambitious the sysop program is: structure, OS branches, sections.
Coverage
What sysop signals the script tries to grab (CPU, disk, mem, and so on).
Polish
Does the report look finished: layout, color, broken fields?
Craft
Clean agent work: fewer turns, tools, and mistakes.
Orphan artifact
A file from a timed-out attempt the scorecard never counted.