What Can a 32GB Mac mini Run Locally? Models, Agents, and Real-World Limits

A 32GB Apple Silicon Mac mini can run useful local models, but “it loads” and “it is a reliable agent backend” are different claims. The practical sweet spot is 8B–14B for low-friction use. Quantized MoE models such as Qwen3.6-35B-A3B can be much faster than a dense model with a similar total parameter count, provided the runtime is optimized for Apple Silicon.

The practical model tiers

Tier	Practical use on 32GB	Typical tasks
8B–9B	Fast and comfortable	chat, summaries, extraction, simple tools
12B–14B	Best default balance	coding help, RAG, structured output, short agents
20B–32B dense	Usable with trade-offs	deeper analysis, low-concurrency work
27B–35B A3B MoE	High upside, tuning required	fast local agents and complex single tasks
70B dense	Not a daily choice	occasional experiments only

A3B means that the model has roughly 3B active parameters per token while retaining a much larger total parameter pool. This reduces per-token computation, but the full quantized weights still need to be stored. MoE is therefore a compute and bandwidth advantage, not a promise that a 35B model needs only 3B of memory.

A useful Apple Silicon setup

Ollama is the easiest starting point. LM Studio is useful for inspecting models, running local APIs, and connecting MCP tools. MLX-based runtimes such as MLX or omlx are preferable when the goal is sustained Apple Silicon inference. llama.cpp remains useful for GGUF models and OpenAI-compatible local servers. Ollama, MLX, llama.cpp, LM Studio

An optimized Qwen3.6-35B-A3B setup

A particularly effective combination on a 32GB M4 machine is Qwen3.6-35B-A3B in 4-bit MLX or a compatible format, served by an Apple-Silicon-native runtime. The useful ingredients are:

1. MoE routing: about 3B active parameters per token.
2. 4-bit weights: lower resident size and memory-bandwidth pressure.
3. MLX/omlx: native use of unified memory and Apple GPU acceleration.
4. MTP/DFlash speculative decoding with a small draft model.
5. A compressed KV cache and bounded tool-result/context sizes.

In the supplied M5 32GB test, this setup reached about 38.4 tok/s at a 1K prompt and 33.1 tok/s at a 4K prompt, with roughly 19GB peak memory. Those figures describe that hardware and configuration; they are not a universal benchmark.

Real-world experience on a 32GB M4 Mac mini

Conversation is generally fine. The experience becomes less reliable when the model must run a long tool loop, execute shell commands, or recover from tool errors. The current practical split is to use local models for private chat, summaries, document work, and low-risk automation, while using online models for serious programming automation.

The tested model list included Qwen3.6-35B-A3B-uncensored-heretic-vision-11mfan46:Q4_K_M, which was comparatively fast and also produced a pelican image.

A tool error such as “A tool message must follow an assistant tool call” usually points to a protocol or integration problem, not simply a lack of model intelligence. Short shell timeouts, incompatible chat templates, missing tool parsers, and weak structured-output constraints can all break an agent loop.

Model ranking by scenario

Scenario	Recommended order
Easy, stable local agent	Qwen3 14B → Qwen3.5 9B → Gemma 3 12B
Fast automation	Qwen3.6-35B-A3B 4-bit → Qwen3-30B-A3B Q4 → Qwen3.5 9B
Chinese tool calls and JSON	Qwen3.6-35B-A3B 4-bit → Qwen3-30B-A3B → Qwen3 14B
Long context and RAG	Qwen3 14B → Gemma 3 12B → Qwen3.6 A3B
Deep reasoning	Gemma 4-26B-A4B → Qwen3.6-35B-A3B → Qwen3 14B
No tuning	Qwen3 14B → Gemma 3 12B → Qwen3.5 9B

Final recommendation

Choose Qwen3 14B when you want a predictable local agent with minimal tuning. Choose Qwen3.6-35B-A3B 4-bit when you are willing to configure MLX/omlx, draft decoding, KV cache, and context limits. A3B improves speed and capability per unit of active computation, but it does not remove the memory and integration limits of a 32GB machine.

Sources

• Apple Mac mini specifications
• Ollama macOS documentation
• Apple MLX
• llama.cpp
• LM Studio documentation