Fabian G. Williams aka Fabs

Fabian G. Williams

Principal Product Manager, Microsoft Subscribe to my YouTube.

I Swapped My Local Coding Model Overnight From a Hotel. The Agent Graded the Upgrade Itself.

Local models are how I run my community work, my volunteer projects, my side hustles, and my musings, for two reasons: cost and keeping client data away from the labs. A new version dropped, so I upgraded it overnight from a hotel, additively, without touching the old one. Then I handed the new model the reviews and let my OpenCode agent test its own upgrade. It stood up a throwaway server, probed itself, and found three config gaps quietly throttling it. Here is the journey, in tables.

Fabian Williams

8-Minute Read

Activity Monitor showing the M3 Max GPU pinned at 92 percent while an OpenCode agent runs probe requests against a throwaway Qwen3.8 test server on port 8082

Local models are how I run my community work, my volunteer projects, my side hustles, and my musings. Two reasons, both simple:

I Made My Evals Replay Every Task on a Local Model. The Frontier Lead Got Thin.

My agents run on frontier models, but a free local model sits idle on a Mac Mini in my office. So I wired my eval system to replay every writing task on the local model and grade both. Across 10 like-for-like rematches the local model reached statistical parity — and beat the frontier model outright on four of them. Here is the receipts-first system that made it prove it.

Fabian Williams

7-Minute Read

Eval cockpit showing the verdict panel: 31 golden cases, 10 replayed, mean like-for-like delta -0.05, with Volume and Quality gates passing

My agents do real work on frontier models. Every dollar of that work is metered against my OpenAI and Anthropic bills. Meanwhile a perfectly capable local model, gpt-oss:20b, sits on a Mac Mini in my office costing me nothing. The obvious question: for which tasks could the free local model do the job just as well?

My Agents Left Me 116 Git Repos. So I Built a Source of Truth.

I let agents build software on my Mac for months, and one morning I could not tell one of them where my own project actually lived. This is the eight hours it took to fix that, and the legend that now keeps my agents honest.

Fabian Williams

8-Minute Read

Stat cards: 116 local repos, 165 on GitHub, 113 logical projects, 3 duplicate clusters, and zero sources of truth becoming one

A few weeks ago I asked an agent to pick up work on one of my own projects. It opened a fresh terminal, looked around, found nothing it recognized, and built a brand new copy of a project I already had. That was not the agent being careless. Nothing on the machine had ever told it where the real one lived. The only map of my own work was a memory in my head, and a memory does not survive contact with a machine where agents do the building.

Recent Posts

Categories

About

Fabian G. Williams aka Fabs Site