Fabian G. Williams aka Fabs

Fabian G. Williams

Principal Product Manager, Microsoft Subscribe to my YouTube.

Doug Was Right: I Swapped In The MoE, And The Concurrency Math Changed

Last post I measured the concurrency ceiling on a 27B dense model and closed with a promise. Doug Ware, who builds applied AI systems, had handed me the one caveat I did not test: a dense model is the hard case, and a mixture-of-experts model with only about 3B active parameters per token should leave real headroom for a second slot to pay off. So I ran the exact same 36-load matrix on an MLX MoE, changed one variable, and let the numbers settle it. The dense model flatlines. The MoE keeps climbing. And at 8 concurrent agents the dense model makes you wait 32 seconds for a first token while the MoE answers in under 1.

Fabian Williams

9-Minute Read

A line chart showing aggregate throughput staying flat near 20 tokens per second on a dense model while a mixture-of-experts model climbs from 58 to 159 tokens per second as concurrent agents rise from one to eight

Two days ago I posted 36 barrier-synchronized loads against 1 local model and let the numbers answer a question Reddit handed me. That post ended with a promise. A reader named Doug Ware, who builds applied AI systems for a living, had replied with the one caveat I had not tested, and I said out loud that measuring it was the next thing I would do. This is that measurement. I did not wait, I did not hand-wave it, and I kept every honest asterisk in.

Recent Posts

Categories

About

Fabian G. Williams aka Fabs Site