LFM2.5-24B-A2B on a Mac mini: 10x faster prefill and 4x faster generation than Qwen 3.8-27B
In the Mac mini LLM series I described the machine I use as a private LLM server and how fast it is with Qwen 3.8 27B: about 100 tokens per second of prompt processing and 30 tokens per second of generation. This post is the follow-up I wanted to write for a very different kind of model.
Liquid AI’s LFM2.5-24B-A2B is a Mixture of Experts model with 24 billion parameters of which only 2.45 billion are active per token. That number is the whole story: a dense 27B model has to stream all 27 billion parameters through memory for every token it generates, while this one only streams the 4 experts that the router picked, out of 64. On the same M4 Pro Mac mini, it is roughly 10x faster at prompt processing and 4x faster at generation than the Qwen model I had been running, in a smaller memory footprint.
It is also a model built for a different job than Qwen: it always thinks before it answers, it was post-trained inside real agent harnesses (OpenClaw, Hermes, pi, OpenCode), and Liquid reports it beating Qwen3.6-35B-A3B on agentic benchmarks even though that model has roughly 50% more total and active parameters.
There is one important negative result, and it is the first thing I checked: LFM2.5 has no MTP head, so the trick that gives Qwen 3.8 a large decode speedup does not apply here. I explain what replaces it below.
The model
| Property | Value |
|---|---|
| Parameters | 24.0B total, 2.45B active |
| Architecture | MoE, 64 experts, top-4 routing, 40 layers (30 conv + 10 GQA) |
| Context | 131,072 tokens (native) |
| Vocabulary | 128,000 |
| Training | 36 trillion tokens, 16 languages |
| License | LFM Open License v1.0 (free below a 10M USD revenue threshold) |
| Released | 2026-10-13 |
Two properties matter for everything that follows.
It is a hybrid model. Only 10 of the 40 layers are regular attention. The other 30 are short-convolution (“recurrent”) blocks that keep a sliding state instead of an attention cache. This is why it is so fast, and it is also the source of the one caching surprise I hit later.
It is a pure reasoning model. Unlike Qwen 3.8, where thinking is a toggle
("think": false in the chat request), LFM2.5 always emits a <think> block
before the answer. The chat template inserts it unconditionally. There is no
mode where the model just answers.
The MTP question, and what replaces it
Qwen 3.8-27B ships with a multi-token prediction (MTP) head: at every step the model drafts a few tokens ahead and the runner verifies them all in one pass. On the Mac mini that is the single biggest decode win, measured at 0.80 to 0.94 acceptance, and it is the difference between roughly 15 tokens per second (the memory bandwidth floor) and the 30 tokens per second I actually get. It needs no draft model and no configuration, the runner just uses it.
So the first question for LFM2.5 was whether it has one. It does not:
- the MLX checkpoint lists 870 tensors and none of them contain
mtpornextn, and the config has nonum_nextn_predict_layersfield; - the model card never mentions MTP;
- for comparison,
mlx-lm’s Qwen3-Next implementation explicitly strips tensors namedmtp., so an MTP head would have shown up in that listing.
What LFM2.5 uses instead is DSpark, a separate draft model published as
LiquidAI/LFM2.5-24B-A2B-DSpark.
It is 327.7M parameters: five attention layers, a rank-256 Markov head that
biases each drafted token towards the one before it, and a confidence head,
with a block size of 9. The target model verifies the whole block in one pass.
For this 24B model, Liquid measures a mean 2.4x throughput improvement on an
NVIDIA B300 at concurrency 1 (up to 3.1x), and 1.7x at concurrency 256; across
the smaller models in the family the blog reports up to 3.18x on GPU and 2.87x
on device.
The catch is where DSpark is supported. I checked all three realistic runtimes on the Mac mini:
| Runtime | Runs LFM2.5? | Speculative decoding |
|---|---|---|
| mlx-lm (direct MLX) | Yes | generic --draft-model only, no DSpark |
| Ollama, MLX engine | No | has DFlash drafters, but no LFM2 at all |
| Ollama, llama.cpp | Yes (GGUF) | only draft-mtp and draft-eagle3 are wired up |
| llama.cpp upstream | Yes | yes: --spec-type draft-dspark |
| SGLang | Yes | yes (--speculative-algorithm DSPARK) |
Two findings are worth spelling out.
Ollama’s MLX engine cannot load LFM2.5 at all. Its registry
(mlxrunner/model/architectures/architectures.go) lists 23 architectures and
the only Liquid one is lfm3. There is no lfm2 and no lfm2_moe. The
lfm2.5:24b model in the Ollama library is GGUF and runs on llama.cpp. The
good news is that the two runners coexist happily: Ollama picks the runner
per model from the manifest, mapping safetensors to MLX and gguf to
llama.cpp, so my Qwen model (safetensors) and LFM2.5 (GGUF) can live in the
same server.
DSpark itself I did not test. Getting it running needs a source build of
llama.cpp plus the 664 MB
LFM2.5-24B-A2B-DSpark-F16.gguf
sidecar, and Liquid’s own on-device numbers are a warning: for the
LFM2.5-8B-A1B they measured only 1.18x speedup on an M4 Max versus 2.54x
on an H100, because verifying k tokens activates more experts and therefore
more weight traffic than a single decode step. For a 24B MoE I would expect
similar or worse. It is the most promising remaining experiment, but it is not
the reason to switch.
Setting it up
The MLX path (the interesting one)
Liquid publishes official MLX builds in 4, 5, 6 and 8 bit. I used the 4-bit
one, LiquidAI/LFM2.5-24B-A2B-MLX-4bit
(13.57 GB), which is the same ballpark as the Ollama GGUF’s Q4_K_M, so the
two are comparable.
One hard requirement: mlx-lm >= 0.32.0, because earlier versions route
the MoE experts incorrectly. The current release is 0.32.2.
export PATH=/opt/homebrew/bin:$PATH
uv venv --python /opt/homebrew/opt/python@3.12/bin/python3.12 ~/lfm25/mlx
uv pip install --python ~/lfm25/mlx/bin/python "mlx-lm==0.32.2"
# 13.57 GB of weights, verified against the Hugging Face checksums
~/lfm25/mlx/bin/hf download LiquidAI/LFM2.5-24B-A2B-MLX-4bit \
--local-dir ~/models/LFM2.5-24B-A2B-MLX-4bitI kept this in its own virtual environment on purpose. The virtual environment
I already use for another model on the same machine has mlx-lm 0.31.3
installed, which is below the 0.32.0 threshold, and upgrading it in place
would have changed the behaviour of a working deployment.
Before loading anything, I checked that the configuration parses and that the weights have the expected layout:
mlx : 0.32.4
mlx_lm : 0.32.2
lfm2_moe : mlx_lm/models/lfm2_moe.py
--- parsed ModelArgs (no weights loaded) ---
model_type lfm2_moe
vocab_size 128000
hidden_size 2048
num_hidden_layers 40
num_experts 64
num_experts_per_tok 4
num_dense_layers 2
num_attention_heads 32
num_key_value_heads 8
conv_L_cache 3
use_expert_bias True
max_position_embeddings 131072
full_attn_idxs [2, 6, 10, 14, 18, 22, 26, 30, 34, 38]That last line is the 10 attention layers among the 40. The checkpoint has 870
tensors and 47 distinct name patterns, all matching the lfm2_moe layout that
mlx-lm expects.
Serving it is one command:
~/lfm25/mlx/bin/mlx_lm.server \
--model ~/models/LFM2.5-24B-A2B-MLX-4bit \
--host 127.0.0.1 --port 8080 --temp 0.1 --top-k 50The server speaks the OpenAI-compatible API, so every client I already have works with it unchanged.
Running it as a service
The Mac is headless, so the server has to be a system launchd daemon rather than a user agent (a user LaunchAgent would only start at login). The plist is a copy of the one I use for Ollama, and it is in the gist together with the pf anchor:
sudo cp com.zonca.mlx-lfm25.plist /Library/LaunchDaemons/
sudo chown root:wheel /Library/LaunchDaemons/com.zonca.mlx-lfm25.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.zonca.mlx-lfm25.plistI bound it to the Tailscale address so the agents on my other machines can reach it, which means adding the port to the pf anchor that keeps everything off the office LAN:
pass in quick on lo0
pass in quick on utun0 proto tcp from any to any port { 22, 11434, 8080 }
block in quick proto tcp from any to any port { 22, 11434, 8080 }One detail worth knowing: the server identifies a local model by its
full path, so requests have to use
/Users/zonca/models/LFM2.5-24B-A2B-MLX-4bit as the model name. The alias
default_model also works. Any other name is treated as a Hugging Face repo
id and comes back as a 404 from the Hub.
curl -s -X POST http://zoncamini:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"/Users/zonca/models/LFM2.5-24B-A2B-MLX-4bit",
"messages":[{"role":"user","content":"What is 17 + 23?"}],
"max_tokens":512}'The Ollama path
Trivial, with one gotcha:
ollama pull lfm2.5:24bThis failed on my machine with
Error: pull model manifest: 412:
The model you are attempting to pull requires a newer version of Ollama.ollama show lfm2.5:24b reports requires 0.40.4, and I was on 0.40.3. The
fix was to upgrade, and since I wanted a single version of Ollama on the
machine anyway, I upgraded in place and dropped the second copy I had
downloaded for the initial pull:
brew upgrade ollama # 0.40.3 -> 0.40.12
sudo launchctl bootout system/com.ollama.serve
sudo launchctl bootstrap system /Library/LaunchDaemons/com.ollama.serve.plistThe launchd daemon points at /opt/homebrew/opt/ollama/bin/ollama, which is
a symlink that Homebrew repoints on upgrade, so the plist needed no editing.
After the upgrade my existing Qwen model still ran on the MLX engine without
any change.
Benchmarks
All numbers are on the M4 Pro with 48 GB of unified memory and about 273 GB/s of memory bandwidth, measured against the same machine’s Qwen 3.8 27B numbers from the setup post.
Prefill and decode against context length
Measured with mlx_lm.benchmark, which times the engine directly with no
server in the way:
| Prompt tokens | Prefill (tok/s) | Generation (tok/s) | Peak memory (GB) |
|---|---|---|---|
| 512 | 961.5 | 115.4 | 14.0 |
| 2,048 | 1064.4 | 112.8 | 14.5 |
| 4,096 | 992.7 | 110.2 | 14.5 |
| 8,192 | 1008.6 | 105.8 | 14.6 |
| 16,384 | 973.3 | 98.0 | 14.8 |
| 32,768 | 873.4 | 85.4 | 15.1 |
| 65,536 | 714.4 | 68.7 | 15.8 |
| 131,072 | 521.7 | 48.4 | 17.2 |
Compare that with Qwen 3.8-27B on the same machine:
| Metric | Qwen 3.8-27B (MLX, NVFP4) | LFM2.5-24B-A2B (MLX, 4-bit) |
|---|---|---|
| Prefill, short prompt | ~100 tok/s | ~1000 tok/s |
| Decode, small context | 26 to 33 tok/s | 105 to 115 tok/s |
| Weights in memory | 18 GB | 13.6 GB |
| Native context | 262,144 | 131,072 |
| Longest context without swapping | 131,072 at 104 tok/s | 131,072 at 522 tok/s |
The prefill number is the one I did not expect. Prefill on a dense model is limited by how fast the machine can stream the weights and apply arithmetic to them: the whole 18 GB weight set has to be read once per token processed, and the M4 Pro’s GPU tops out around 100 tok/s of prefill for a 27B dense model. (Decode has it even worse, since it needs one full pass over the weights per token, which is the 15 tok/s bandwidth floor that MTP doubles.) The MoE changes both sides of that equation, because a token only touches 4 of the 64 experts, so both the weight traffic and the arithmetic per token drop by roughly an order of magnitude. Prefill stays above 500 tok/s even at the full 131K context, and the machine never swapped: peak memory was 17.2 GB out of 48 GB.
Decode degrades with context, from 115 tok/s at 512 tokens to 48 tok/s at 131K, because the attention layers have more to read from the KV cache. That is the same shape as the Qwen curve, just shifted up by 3 to 4x.
Through the server API
The numbers above are the engine. Through the OpenAI-compatible API, with the overhead of HTTP and token streaming included:
| Test | TTFB | Wall | Output tok/s |
|---|---|---|---|
| Small prompt (5 tokens), 256 tokens out | 0.18 s | 2.68 s | 95.5 |
| 10,800-token prompt, cold | 10.80 s | 12.26 s | (prefill at 1000 tok/s) |
| 10,800-token prompt, warm | 0.04 s | 1.48 s | 86.2 |
| 800-token prompt, predictable content, 512 out | 0.79 s | 5.92 s | 86.4 |
| 1,163-token prompt, random hex, 512 out | 1.11 s | 6.28 s | 81.5 |
MLX versus llama.cpp
The same model as a GGUF through Ollama 0.40.12 (llama.cpp, Q4_K_M):
| Test | MLX (mlx-lm) | llama.cpp (Ollama) |
|---|---|---|
| Cold model load | ~2 s | 10.2 s |
| Prefill, 65,539-token prompt | 1000 tok/s (MLX server) | 570.7 tok/s |
| Decode, predictable content | 86.4 tok/s | 90.7 tok/s |
| Decode, random content | 81.5 tok/s | 81.8 tok/s |
| Context window | 131,072 | 131,072 |
Decode is essentially a tie between the two backends, and llama.cpp is slightly better on it. Prefill is clearly better on MLX (1000 against 571 tok/s). The model load is much faster on MLX. So the reason to prefer MLX here is prefill and cache behaviour, not decode.
One note on the context window: my launchd plist sets
OLLAMA_CONTEXT_LENGTH=200000 globally, for the Qwen model, and Ollama
correctly clamps that down to the 131,072 that LFM2.5 actually supports.
The caching surprise
This is the one result that surprised me, and it is a direct consequence of the hybrid architecture.
llama.cpp keeps a prompt cache and, for pure attention models, reuses the processed prefix across requests. For LFM2.5 the exact-prefix case works fine: 41,052 tokens, sent twice, the second request reuses 41,048 of them and the prompt evaluation drops from 58.8 s to 0.05 s.
But a partial prefix match, which is what a growing conversation looks like, does not work as well:
| Case | llama.cpp | MLX |
|---|---|---|
| Cold, 41K tokens | 58.8 s, 0 cached | 51.5 s, 0 cached |
| Exact repeat | 0.05 s, 41,048 cached | 0.34 s, 41,041 cached |
| Same prefix, different tail | 4.06 s, 39,004 cached | 0.38 s, 41,041 cached |
| Growing prefix, turn 1 | 3.95 s, 39,009 cached | 0.40 s, 41,041 cached |
| Growing prefix, turn 2 | 3.97 s, 39,020 cached | 0.42 s, 41,057 cached |
| Unrelated 96K prompt | 210.2 s, 0 cached | 166.4 s, 0 cached |
On MLX a multi-turn conversation only reprocesses the handful of new tokens. On llama.cpp every turn reprocesses about 2,000 tokens, and the log says why:
forcing full prompt re-processing due to lack of cache data
(likely due to SWA or hybrid/recurrent memory, see
https://github.com/ggml-org/llama.cpp/pull/13194)The short-convolution layers carry a recurrent state that llama.cpp cannot checkpoint and restore, so when the cached prefix does not align with a usable checkpoint, it falls back to re-running the prompt from the last checkpoint it can use. In an interactive agent that is 4 seconds of prefill per turn that you do not need to pay.
For an agent doing multi-turn work, that makes the MLX server the better choice even before the raw speed numbers.
Reasoning quality
I ran the same 10-question suite I used to compare Qwen 3.6 against Qwen 3.8, with thinking on and a 200-token budget, then re-ran whatever ran out of budget with 2,000 tokens:
| LFM2.5 (llama.cpp) | LFM2.5 (MLX) | Qwen 3.8-27B | |
|---|---|---|---|
| Score at a 200-token budget | 6/10 | 5/10 | 8/10 |
| Score after the 2,000-token re-run | 9/10 | 9/10 | 10/10 |
The pattern is the same one that showed up when I compared Qwen 3.6 to 3.8: the model reasons well but overthinks trivial questions. With a tight budget, LFM2.5 spends the tokens thinking and never emits the answer. Give it room and it gets them right, with one exception: the “a farmer has 3 horses, how many legs in total” question, where the llama.cpp path never answered at all and the MLX path answered 14 (wrong, it should be 12).
So on this small suite Qwen 3.8 is still ahead, mostly because it is more concise. That is a real trade-off for an agent: less thinking means faster turns and a smaller context.
Using it as an agent
The context an agent burns on its first request is the real cost of using a local model, so I measured the same agents I had measured against Qwen. They all talk to the same Ollama endpoint, so pointing them at LFM2.5 needed no configuration changes:
| Agent | Version | Context (tokens) | Time to first answer | Prefill | Decode |
|---|---|---|---|---|---|
| pi | 1.4.0 | 2,839 | 5.6 s | 952 tok/s | 86.7 tok/s |
| goose | 1.54.0 | 4,673 | 19.6 s | 978 tok/s | 91.3 tok/s |
| opencode | 2.2.0 | 6,224 | 51.8 s | 962 tok/s | 84.4 tok/s |
| pi | 1.2.0 | 2,828 | 29.1 s | ~100 tok/s | ~30 tok/s |
| goose | 1.54.0 | 4,716 | 55.3 s | ~100 tok/s | ~30 tok/s |
| opencode | 1.19.0 | 8,956 | 103 s | ~100 tok/s | ~30 tok/s |
(The lower three rows are the Qwen 3.8 numbers from the setup post, for reference.)
The context sizes are essentially identical, as expected, since they are a property of the agent’s system prompt and tool definitions rather than of the model. The times are not: pi’s first answer goes from 29 seconds to under 6, and opencode’s from 103 seconds to 52. The 30 second startup I complained about in the experience post is now a 5 second startup, mostly because the preamble is preprocessed ten times faster.
Tool calling works. I asked pi to read a file and report what was in it:
> Use your read tool to read /tmp/lfm25_tooltest.txt and then tell me exactly
what the magic word and number are, in one short line.
The magic word is **purple elephant** and the number is **42**.That was 2.4 seconds on a warm cache. In an earlier attempt the agent was pointed at a file that did not exist on its own filesystem, and it said so instead of making something up, which is the behaviour I want.
What I would use it for
LFM2.5-24B-A2B, on MLX, if you want speed and you can live with the thinking. The numbers are not close: 10x the prefill and 4x the decode of the dense model, in less memory, with a cleaner multi-turn cache. For an agent whose cost is dominated by re-reading a large context, that is the whole ballgame.
Qwen 3.8-27B if you want concise answers or better judgement on small reasoning tasks. It scored higher on my 10-question suite, and it does not spend 700 characters of thinking on “what is 17 plus 23”.
Both, if you can run two servers. They have complementary strengths: the Qwen model is the more careful reasoner, LFM2.5 is the one that keeps up with an interactive agent.
And DSpark is still on the table. The 1.18x that Liquid measured on-device for the smaller MoE is a warning, not a verdict, and I would like to measure it properly. That needs a source build of llama.cpp with the DSpark sidecar, which is a weekend project rather than a five-minute install.
Notes on running two models at once
The Mac mini has 48 GB of unified memory, and the two models are 18 GB and 14 GB. That does not leave much room for two KV caches at long context, so running both at once means accepting a small context window on each.
What I did instead, and would recommend, is run one at a time and unload the
other with ollama stop. Ollama picks the runner per model, so the same
server handles the MLX model (Qwen, safetensors) and the llama.cpp model
(LFM2.5, GGUF) without any reconfiguration.
One last operational note: the Ollama server binds to the Tailscale address only, and so does the MLX server, so nothing is exposed on the office LAN. The port 8080 rule above is what keeps it that way. All of this is covered in the setup post.
Reproducing this
The benchmark scripts are in this gist:
bench_api.py: the MLX server API benchmark (TTFB, prefill, decode, cache)bench_ollama.py: the same through Ollama’s native/api/generatebench_reasoning.py: the 10-question reasoning suite, Ollama and MLX backendstest_cache.py: the prefix cache behaviour in both backendscom.zonca.mlx-lfm25.plist: the launchd daemon for the MLX serverpf.anchors.com.zonca.macmini: the pf anchor that opens port 8080 on the tailnet only
They only need the standard library plus transformers for the tokenizer, so
they run with the same mlx-lm virtual environment:
~/lfm25/mlx/bin/python bench_api.py \
--model ~/models/LFM2.5-24B-A2B-MLX-4bit \
--out results.json
~/lfm25/mlx/bin/python bench_ollama.py \
--model lfm2.5:24b --host http://100.91.172.9:11434 \
--out results.json
~/lfm25/mlx/bin/python bench_reasoning.py \
--backend ollama --model lfm2.5:24b --out reasoning.json