L0 — Engine bake-off

Same model across every engine (Qwen3-Coder-30B-A3B), box single-tenant, matched bit-budget (MLX-4bit vanilla ≈ GGUF-Q4_K_M, ~4.5 bpw; DWQ shown as MLX's best). Two phases because the best config differs by regime. ← [[Benchmarking a whole local-LLM stack on an M4 Pro — and auditing my own mistakes|back to overview]]

Single-stream (1 user) — MLX wins, and not because of DWQ

engine	quant	decode t/s	prefill@4k	prefill@32k	wired GB
mlx_lm	4bit	89.3	848	757	19.9
mlx_lm	4bit-DWQ	90.3	843	733	19.9
llama.cpp	Q4_K_M	70.7	789	568	25.8
ollama	Q4_K_M	65.0	718	354	24.5
LM Studio	Q4_K_M	57.3	708	534	24.8

MLX is ~26% faster decode and ~6 GB leaner than llama.cpp at the same bit-budget — the win is the engine, not the quant.

Under concurrency — the inversion

type: line
title: System throughput vs concurrent requests (tok/s)
x: ["1", "2", "4", "8", "16"]
series:
  - name: llama.cpp
    data: [63, 86, 98, 105, 134]
  - name: LM Studio
    data: [63, 87, 96, 96, 96]
  - name: ollama
    data: [37, 64, 80, 89, 101]
  - name: mlx_lm
    data: [78, 82, 82, 78, 76]

[!IMPORTANT] This corrects two earlier wrong claims. MLX does batch (its log shows 10 concurrent sequences) — so my original "MLX can't batch" was a config artifact (I'd left concurrency disabled). But MLX's batched decode yields no aggregate-throughput gain (flat ~80 t/s), while llama.cpp's scales to 134. The accurate statement: MLX batches, but batching doesn't raise aggregate throughput here; llama.cpp's does.

Verdict: single-user → MLX (fastest, leanest). Multi-user aggregate throughput → llama.cpp.

Soak / thermal & speculative decoding

No thermal throttling on 8-min sustained decode (mlx 90.3→90.0, llama.cpp 70.7→70.6 t/s) — burst ≈ sustained on this M4 Pro.
Speculative decoding is net-negative for the A3B MoE: a Qwen3-0.6B draft slowed DWQ-30B from 92.6 → 59.4 t/s (0.64×). The MoE activates only ~3B params/token, so draft+verify overhead dominates. Don't enable it for A3B MoE.