Choosing your stack — pros, cons & a decision guide

A practical "pick X if…" guide derived from the results. ← [[Benchmarking a whole local-LLM stack on an M4 Pro — and auditing my own mistakes|back to overview]]

[!NOTE] These calls are for this box (M4 Pro, 64 GB, single-user dev) and coding/agentic workloads. The reasoning transfers; the exact numbers won't.

TL;DR — scenario → stack

:::tabs

Daily agentic coding (best value)

mlx_lm.server + Qwen3-Coder-30B-A3B-4bit-DWQ + aider or goose. Fast, lean, reliable tools. This is the default.

Maximum solve rate (cost no object)

Same engine/model, but claude-code as the harness — it lands tasks the lean harnesses miss, at 15–50× the tokens/time. Use when you'll pay for it.

Lowest memory

gpt-oss-20b (12–17 GB) if you don't need tool-calling, else DWQ-4bit (20 GB). Skip 6/8-bit and the 80B.

Multi-user / serving others

llama.cpp (it batches — 134 t/s at 16 concurrent vs MLX's flat 80), same DWQ model.

Tool-using agents

Qwen3-Coder family on mlx — only. Every other model's tool-calling is a serving-format minefield here.

RAG / embeddings

qwen3-embedding:0.6b — fast, perfect recall on the test set, tiny. :::

Engine — pick one

graph TD
  A{Serving just yourself?} -->|yes, on a Mac| B[mlx_lm.server]
  A -->|need to serve many| C[llama.cpp]
  A -->|want easiest setup| D[ollama]
  A -->|want a GUI| E[LM Studio]
engine pros cons
mlx_lm.server fastest single-stream, leanest memory, native Qwen3 tool-parsing, trivial setup no throughput gain under load; Mac-only; one model per server (requesting another id silently swaps it); doesn't parse gpt-oss/Mistral tool formats
llama.cpp best batching/throughput, runs everywhere, broadest model+quant support, --jinja parses many tool formats ~25% slower single-stream than MLX; one tool_call per response; fiddlier flags
ollama easiest UX, model registry, auto-manages slower; batching off by default (NUM_PARALLEL); long-context degraded vs MLX
LM Studio nice GUI, runs MLX and GGUF, llama.cpp-class throughput wrapper overhead, higher cold TTFT, GUI-centric

Model & quant — pick one

graph TD
  M{Need tool-calling / agents?} -->|yes| Q[Qwen3-Coder family]
  M -->|no, pure codegen| Q
  Q --> R{Memory budget?}
  R -->|~20 GB, the sweet spot| DWQ[30B-A3B-4bit-DWQ ✅ default]
  R -->|<17 GB, no tools needed| OSS[gpt-oss-20b]
  DWQ -.->|don't bother| MORE[6/8-bit or 80B: more cost, no quality gain]
model / quant pros cons verdict
30B-A3B-4bit-DWQ best HumanEval+ (0.93), tools ✓, 32k ✓, 90 t/s, 20 GB — default pick
30B-A3B 6/8-bit marginally different 1.3–1.8× memory, ~half the speed, no quality gain skip
30B-A3B 4-bit (plain) same speed/mem as DWQ lower quality (0.86) + weaker tools use DWQ instead
Coder-Next 80B flagship lower HE+ than DWQ-4bit, 2× memory, slower not worth it
gpt-oss-20b smallest (12–17 GB), codes well tool-calling + long-context unusable via local serving only if RAM-bound & no tools
dense (qwen2.5-32B, devstral-24B) qwen2.5 codes very well 5–13× slower (7–13 t/s) too slow for interactive

Agent harness — pick one

harness pick if… pros cons
aider you want efficient, scriptable edits leanest tokens, fast, robust edit formats one-shot; fewer hard-task solves
goose efficient + lightly agentic leanest + fastest, uses tools lower solve rate on hard tasks
claude-code you need the most solves & will pay highest solve rate via iteration, great UX 15–50× tokens/time; never self-terminates on a local model; needs the Anthropic→OpenAI shim
crush you want an agentic alternative capable spirals to 100–300k tokens, timeouts
opencode interactive daily use (your lcld) solid interactively heaviest (up to 700k tok/task) + flaky headless

Honest caveats

[!CAUTION]