The new model
Liquid AI released LFM2.5-2.6B days ago: a dense 2.6B hybrid (mostly gated convolutions, eight attention layers), built for on-device use — it runs in about 2.5 GB of memory. We pointed our frozen evaluation instrument at it — the same generation harness, grader, and sampling parameters as all our banked runs, unchanged by commit hash — measuring the model alone, then the model plus GoM verified selection. Nothing about the machinery was tuned for it.
The result
| suite | LFM2.5-2.6B alone (avg pass@1) | + GoM verified selection@8 | gemma-4-26B-A4B (MoE) single-shot | gemma-4-31B (dense) single-shot |
|---|---|---|---|---|
| MBPP+ (378) | 71.1% | 80.4% | 79.7% | 79.4% |
| HumanEval+ (164) | 83.1% | 91.5% | 91.6% | 91.9% |
On MBPP+ — the suite where our 4B attempt fell 4.5 points short — the 2.6B with verification beats both bigger models' single shots outright, and ties the 31B's own verified-selection score. On HumanEval+ it lands within a tenth of a point of the 26B and four tenths of the 31B: statistically indistinguishable (the confidence intervals overlap almost entirely; a second seed is running as we post).
The MBPP+ result is replication-hardened: two independent seeds (80.4% / 80.2%) plus a higher-budget pass that clears our strictest truncation guard (80.2% at 1.4% token-capped) — three concordant runs, every one above both bigger models' single shots. HumanEval+ is reported as what it is: statistical parity across seeds (91.5% / 89.6%), not a win.
On the model's own showcase benchmarks
Liquid's card leads with agentic and reasoning benches. We re-ran the two we could execute honestly in our own harness — both arms, same instrument; their published numbers appear only as reference (different harness, different settings — the AIME gap shows why that distinction matters):
| benchmark (theirs) | LFM2.5 alone | + GoM selection | selection type | fixed / broke | vendor-reported* |
|---|---|---|---|---|---|
| LiveCodeBench v6 (n=300) | 46.3% | 57.3% (+11.1) | verified (code executed) | 32 / 0 | 59.4 |
| AIME25 (n=30) | 27.5% | 33.3% (+5.8) | majority vote (no verifier) | 3 / 2 | 51.9 |
*their harness and settings, shown for reference only — never compared on the same bar.
Those two rows carry the deepest finding in this note. On the same 300 LiveCodeBench problems, with the identical eight samples per task, verified selection scores 57.3% while majority voting scores 47.0% — verification beats voting by ten points on the same raw material. And the guarantee asymmetry: across every suite where an exact verifier exists, selection fixed 158 tasks and broke zero (~1,900 task-instances). Where only voting exists, it broke 2 of 30. The verifier is where the value lives — which is why GoM's machinery is built around owning it.
It reproduces on a laptop
We replayed the same protocol on an Apple M5 Max — Liquid's own MLX build, ~86 tokens/second, the whole stack in memory a phone could afford. On the first half of HumanEval+ (the clean half of a two-machine split; the full-suite on-device run completes tonight): 88.3% → 93.9% with selection, oracle 95.1%, nothing broken. The selection dynamics — near-total capture of the candidate pool's ceiling, do-no-harm intact — reproduce on-device.
What we are NOT claiming
Three things, stated as plainly as the wins.
Not cheaper per query. LFM2.5 writes long, deliberate completions; eight samples cost roughly 5× the FLOPs of one 31B pass. The claim is capability per footprint, not per token: a ~2.5 GB model doing what previously took a ~62 GB one — the difference between "runs on the vehicle" and "runs in a datacenter."
Not our model. LFM2.5 is Liquid AI's work and it is excellent. GoM is the machinery around a frozen model — verification, selection, routing, learning — and it is deliberately model-agnostic. A better base model makes the machinery more valuable, not less: verification turned this one's 71% into 80%.
Not a pre-registered result. This is an exploratory measurement on our frozen instrument — no pre-registered bars — and we label it that way, publishing the whole ledger either way.
Why this matters
The interesting curve here isn't any single model. It's that the verifier is the durable asset. Base models improve monthly and the improvements are free to whoever has the machinery to harvest them: the same untouched selection harness carried a days-old 2.6B past a dense 31B's single-shot. Own the verification-and-learning layer, and every model release — anyone's — compounds in your favor.
Protocol: k=8, temperature 1.0, top-p 0.95, 8192-token budget, thinking-off parity; evalplus HumanEval+ v0.1.10 / MBPP+ v0.2.0, plus-tier verdicts; selection screens on base-tier tests, verdicts always on held-out plus tests. Comparator numbers measured 2026-08-06 on 2×H200; LFM2.5 measured 2026-08-10 on a B300 sliver — same engine family and parameters, different silicon, stated. LFM2.5-2.6B under the LFM Open License v1.0. Full tables and receipts in the research ledger.