Mac mini as a private LLM server: the technical setup
This is the technical companion to the experience post: how the Mac mini is set up, how every service runs, and where the configuration lives. The benchmark details (prefill, decode, context stress test, agent context sizes) are here too. The data-center comparison is in the comparison post.
The machine and the model
- Hardware: Apple M4 Pro, 48 GB unified memory, headless (no GUI login session, only SSH + Tailscale).
- Server: Ollama 0.40.3 with the Apple MLX engine.
- Model: Qwen 3.8 27B NVFP4, 18 GB, 262 K native context, vision, MTP head. Upstream nvidia/Qwen3.8-27B-NVFP4 (quantized from Qwen/Qwen3.8-27B, Apache 2.0).
NVFP4 is a 4-bit quantization format: instead of the ~54 GB the model would need in bf16 (2 bytes per value), the weights take 18 GB, which is what makes a 27B model practical on a 48 GB machine, with room left for the context memory. The model also ships a multi-token prediction (MTP) head: at every step it drafts a few tokens ahead and the runner verifies all of them in a single pass. When the drafts are right (measured acceptance is 0.80-0.94), generation advances by 2-3 tokens per step instead of 1, which is why decode runs around 30 tok/s rather than the ~15 tok/s the memory bandwidth alone would allow. No draft model is needed, and an MTP head cannot stack with a separate drafter.
Two phases, one bottleneck
Serving a prompt has two phases, and it helps to keep them separate when reading the benchmarks. Prefill processes the input: the whole prompt is pushed through the model in parallel batches, so a long prompt takes proportionally longer. Decode then generates the answer one token at a time; each generated token needs one full pass over the model weights, so decode speed is capped by how fast the machine can stream those 18 GB from memory (the M4 Pro moves ~273 GB/s). Along the way the server builds the KV cache, a per-token memory of everything the model has attended to: it grows with the context, which is why a bigger context window costs memory even when it is empty, and why the largest windows can push a 48 GB machine into swapping.
Running Ollama as a service on a headless Mac
The usual advice is brew services start ollama, but a Mac without a GUI login
has no gui/501 launchd domain, and brew services fails with
launchctl ... Domain does not support specified action (error 125). A user
LaunchAgent only starts at login. The reliable solution is a system launchd
daemon that runs as user zonca at boot:
/Library/LaunchDaemons/com.ollama.serve.plist, withRunAtLoadandKeepAlive; logs go to/tmp/ollama.logand/tmp/ollama.err.log.- Tuned environment:
OLLAMA_FLASH_ATTENTION=1(memory-efficient attention)OLLAMA_KV_CACHE_TYPE=q8_0(8-bit KV cache, half the memory of 16-bit)OLLAMA_KEEP_ALIVE=48h(the model stays resident)OLLAMA_CONTEXT_LENGTH=200000(practical limit on 48 GB, see below)OLLAMA_LLM_LIBRARY=mlx_metal_v3(the Apple MLX engine)
Manage it with sudo launchctl bootout system/com.ollama.serve and
sudo launchctl bootstrap system /Library/LaunchDaemons/com.ollama.serve.plist.
Note that kickstart -k restarts the process but does not re-read a modified
plist: after an edit, use bootout plus bootstrap.
Not a single open port, except over Tailscale
The mini sits on the office LAN, where port 22 and 11434 should not be reachable. Two mechanisms do that:
- Ollama bound to the tailnet address:
OLLAMA_HOSTset to the node’s Tailscale IP with port 11434 in the plist, so the server listens only on the Tailscale interface. - pf firewall for SSH: macOS sshd is launchd socket-activated and always
creates a
*:22socket (itsListenAddressin sshd_config is ignored). A pf anchor allows ports 22 and 11434 only on the tailnet interfaceutun0(plus loopback) and blocks them everywhere else. macOS does not enable pf automatically, so a second daemon (com.zonca.pfctl-enable) runspfctl -eat boot.
A listener audit then shows only launchd *:22 (pf-filtered), Ollama bound to
the Tailscale address on port 11434, and Tailscale’s own sockets.
Sharing with colleagues: Tailscale device sharing (admin console, Machines, Share this machine) gives each invited person access to only the Mac mini, with end-to-end encryption, quarantined direction (responds to inbound only), and instant revocation. For Ollama the OpenAI-compatible endpoint just works: point the client at the tailnet hostname with port 11434.
Benchmarks (2026-10-13)
Measured directly against Ollama’s API (/api/generate), model warm, memory
bandwidth bound. Context during the benchmark: the server is configured with
OLLAMA_CONTEXT_LENGTH=200000, while the model natively supports 262,144
tokens.
| Metric | Result |
|---|---|
| Prefill, 2039-token prompt | 20.1 s (~101 tok/s) |
| Prefill, 4044-token prompt | 40.3 s (~100 tok/s) |
| Decode, predictable prose, 256 tokens | 28.7 tok/s |
| Decode, random hex, 512 tokens | 32.7 tok/s |
| MTP acceptance (measured across runs) | 0.80-0.94 |
| Cold model load (after full unload) | ~4 s |
Prefill sits at ~100 tok/s for short prompts: the M4 Pro memory-bandwidth floor for a 27B dense hybrid. At very long contexts it degrades, which is what the stress test below shows.
Context stress test
I stress-tested the context limit with growing prompts while watching memory and swap:
| Context | Prefill | Result |
|---|---|---|
| 65 K | 78.6 tok/s | no swap, fine |
| 131 K | 104.5 tok/s | no swap, fine |
| 192 K | ~60 tok/s | no swap, fine |
| 262 K (native max) | collapses to ~25 tok/s | swaps up to ~6 GB, unusable |
So 200 K is the practical default on 48 GB: no swapping, and the only cost of the larger window is slower prefill and generation at long context.
The agent benchmark that matters: context size
For a coding agent the real cost is how much context it burns on its first request: on this machine, 10 K tokens of preamble means an extra minute of waiting per new session. Same trivial question, first-request token count as seen by Ollama:
| Agent | Version | First request context | Time |
|---|---|---|---|
| pi (pi-coding-agent) | 1.2.0 | 2,828 tokens | 29 s |
| goose | 1.54.0 | 4,716 tokens | 55 s |
| opencode | 1.19.0 | 8,956 tokens | 103 s |
| crush | 0.98.1 | 12,950 tokens | 120 s |
pi is the lightest: built-in read/bash/edit/write tools, support for Agent
Skills (folders with a SKILL.md of instructions that the agent can discover
and load on demand; an open standard shared by several agents), MCP (a
protocol for plugging external tools and services into an agent), and still
under 3 K tokens of initial context, so the first answer comes in about half a
minute instead of two. goose has tool calling and extensions but not the Agent
Skills standard; crush supports both but its context is larger than
opencode’s. For a local model, context size is the bottleneck, so pi wins.
Configuration files
The complete configuration (launchd plist, pf anchor, pf-enable daemon, and the OpenAI-compatible provider configs for pi, goose, and crush) is in this gist:
Once all of this runs, the natural question is how the little machine compares against real data-center hardware serving the same model: that is the next post.