What Meta shipped
Meta released Muse Glimmer 30B today: an open agentic model under Apache 2.0, with a four-level reasoning dial (low → xhigh) and a launch chart that compares it against gemma-4-31B and Qwen3.6 — two model lines we have already instrumented end-to-end in this series. Meta’s recommended sampling (temperature 1.0, top-p 0.95) happens to match our frozen protocol almost exactly.
Nothing could even run it yet
Here’s what day zero actually looked like. The released serving stack didn’t know the architecture — the checkpoint literally pins an unreleased library build. The high-throughput engines produced garbage. There were no serving recipes, no tuned prompts, no folklore to lean on. So we went underneath: loaded the reference implementation directly, put our standard OpenAI-compatible interface in front of it, and ran the exact same battery we run on everything — same sampler, same grader, same selection rule, same held-out verdicts. Slower and plainer than a production serve, and that’s the point: when a model is too new for the ecosystem, the verification machinery still works. By evening, a checkpoint that nothing could load at breakfast had fully graded, execution-verified results.
First numbers — most conservative setting
First numbers, generated at the model’s lowest reasoning setting (its fastest mode — the closest available analogue to the thinking-off parity arm used throughout this series), k=8 samples, seed 0, zero truncated samples on either suite:
| suite | Muse Glimmer 30B alone (avg pass@1) | + GoM verified selection@8 | oracle ceiling | fixed / broken |
|---|---|---|---|---|
| HumanEval+ (164) | 85.7% | 89.0% (+3.3) | 90.9% | 2 / 0 |
| MBPP+ (378) | 71.7% | 74.3% (+2.6) | 79.6% | 13 / 0 |
The row that matters most is the last column. Across all 542 problems, selection fixed 15 tasks and broke zero — pairwise against the model’s own single shot, on a checkpoint that was hours old. That is the same structural guarantee we measured on NVIDIA’s Nemotron line and on Liquid AI’s LFM2.5: selection screens candidates on visible tests and is graded on held-out ones, so where an exact verifier exists, it can win but not lose. Three model families, three release eras, one mechanism.
What’s still running tonight
Two arms are in flight as this note goes up: a second seed of the conservative arm (replication before any number hardens), and the default-strength arm — the reasoning level Meta’s own launch numbers use. That arm is the fair one for any cross-model comparison, and it is where the vendor-axis chart (Muse Glimmer ± GoM beside gemma-4-31B and Qwen3.6, the exact models Meta chose for its launch chart) will come from. This section will carry those results when they land; nothing above changes either way.
What we are NOT claiming
No comparison claims yet. The conservative arm deliberately handicaps the model — at its lowest reasoning setting, Muse Glimmer + GoM does not clear the gemma-4 single-shot scores on these suites, and does not clear the smaller LFM2.5 + GoM result on MBPP+ from the previous note. If the default-strength arm changes that picture, the numbers will say so; if it doesn’t, they’ll say that too. We are also not claiming the lift is large — +3.3 and +2.6 are real but modest, measured with zero truncation and confidence intervals in the ledger. The claim tonight is narrower and, we think, more valuable: the machinery transfers to a day-zero checkpoint unchanged, and its safety property holds there.
Why this matters
New models now land weekly, and every serious operator faces the same two questions within hours: is this one better for our workload — and what does it break? Most teams wait weeks for the ecosystem to settle before they can answer. We answered same-day: graded, execution-verified, regression-audited numbers before midnight, on serving infrastructure that did not exist for this model when the announcement went up. And the pattern that emerged is the one we keep measuring everywhere: put a real verifier next to a frozen model and you get gains for free — on a checkpoint that is hours old, exactly as on a two-year-old flagship. That turnaround, not any single score, is the capability.
Protocol: k=8, temperature 1.0, top-p 0.95, seed 0; reasoning_strength=low with the final-answer channel forced (the model has no true thinking-off; stated); 12,288-token budget, 0.00% samples at cap on both suites; evalplus HumanEval+ v0.1.10 / MBPP+ v0.2.0, plus-tier verdicts; selection screens on base-tier tests, verdicts always on held-out plus tests. Serving: reference transformers implementation (development build, as the checkpoint requires) behind an OpenAI-compatible shim on B300 hardware; weights via the Unsloth repack of Meta’s release (their pinned build identified the requirement); the standard high-throughput stack lacked native support at measurement time. Muse Glimmer 30B under Apache 2.0. Full tables, confidence intervals, and receipts in the research ledger.