A note on posture before the numbers: this is not an NVIDIA collaboration, and nothing here should be read as their endorsement. We used the Nemotron models exactly as publicly released, froze them, and measured. We publish this because the results say something we think matters to anyone who ships or deploys foundation models: where the next factor of capability comes from once the weights are already trained.
550B: the flagship, composed — +69% at the same reasoning budget
ARC-AGI-2 is the abstract-reasoning benchmark frontier labs currently use to demonstrate progress, and it is brutally resistant to memorization. To our knowledge, no published ARC-AGI-2 score exists for Nemotron's 550B flagship — so we measured one, both ways.
Answering directly at a generous 64k-token reasoning budget, the frozen model solves 13 of the 120 public-evaluation tasks. That number alone corrects the record: our own earlier runs had it at zero, and the zero turned out to be an artifact of starving it — at a 16k budget its answers were cut off mid-thought so often that only 6.5% even parsed; at 64k, 87.4% parse and 13 tasks fall. The model was never incapable. It was under-budgeted.
Then the same frozen model, with the same 64k-token budget on every attempt, working inside GoM's machinery — write a small program instead of guessing a grid, run it, check it against the worked examples, repair what fails, select among independently derived candidates that survive: 22 of 120.
| Configuration | ARC-AGI-2 public eval (our dev set; self-run) |
|---|---|
| Frozen 550B, direct answering — earlier 30-task probe at a 16k budget (retired as budget-starved) | 0 / 30 |
| Frozen 550B, direct answering, 64k budget | 13 / 120 |
| Frozen 550B inside GoM's write-verify-repair-select loop, same 64k per-attempt budget | 22 / 120 (+69%) |
The two modes also solve different puzzles — counting every configuration tried across the whole campaign, about 28 of the 120 fell cumulatively, including tasks each mode alone finds impossible. (The external number is the 22; the 28 is campaign-cumulative.) That is the composition thesis in one line: the machinery isn't a better model, it's a way to get more out of the one you already have.
120B: a brain attached to Super — +23 points, verdict in days
One rung down the ladder, the experiment changes shape: instead of wrapping the model, we attach to it. A small GoM-Brain reads the residual stream at one interior layer of the frozen 120B Super and writes back a nudge — the model's weights stay untouched; the brain is the only thing that trains.
The onboarding ran our standard gated protocol: scan for the right attach layer, train a ladder of checkpoints, evaluate every rung, and let pre-registered guards — not enthusiasm — pick the winner. The headline, at confirmation sample size: base 53.5% → brain 76.5% on GSM8K (n=200 per arm, +23.0 points), with the peak corroborated by adjacent checkpoints rather than standing alone. A four-benchmark multiple-choice harm check (n=500 each) came back clean within noise on seven of eight cells, with the eighth flagged for the formal review.
A calibration note an informed reader deserves: these are deliberately conservative full-forward, zero-shot harness scores, and they sit well below the scores NVIDIA publishes for these models under their intended chat-mode configurations. The claim here is the method-matched delta — both arms measured under identical conditions — not a statement of the product's ceiling. Family evidence scopes the delta further: the brain repairs a weak reasoning path rather than stacking on a strong one (on a base already scoring 90%+ under this harness, the same attach measures as a wash), and both Nemotron bases sit squarely in the repair regime under this measurement.
Two details we'd want to see if we were reading this from the outside. First, the guard story: one early checkpoint scored deceptively well while its internal health signal had collapsed — the exact failure shape that fooled us once on a sibling model. This time the pre-registered guard rejected it automatically, before selection. Second, the label: this result is interim — the formal pass/fail verdict against pre-registered bars lands this week, and we publish the outcome either way.
Nano: the edge class — onboarded in an afternoon
The 3B-active Nemotron Nano is the interesting one for where we think inference is going. It's also a hard test of generality: a hybrid Mamba-2 + mixture-of-experts design — an architecture class nothing like the transformer lines the brain was proven on first.
| Benchmark | Frozen Nano | + GoM-Brain | Δ |
|---|---|---|---|
| GSM8K (grade-school math) | 25.0% | 37.5% | +12.5 (p≈0.007) |
| ARC-Challenge | 45.8% | 51.6% | +5.8 (trend, p≈0.06) |
| ARC-Easy | 75.4% | 79.4% | +4.0 (trend) |
| PIQA | 81.6% | 82.8% | +1.2 (noise) |
GSM8K is the significant result; the multiple-choice rows are individually within noise and are best read the way we read them internally — a no-harm measurement with a positive trend, not four separate wins. The brain is 376M parameters and trained in under two hours on a single GPU. Nothing in the row got worse. That is the whole onboarding cost for a new model family — an afternoon, one GPU, no access to the base model's training data or recipe. (Regular readers will recognize this base from Note 2, where an early ~40 MB prototype attach was evaluated under a custom harness that scores higher in absolute terms; this row is a different, larger brain re-measured under the standard public harness — the before/after delta within one harness is the comparison.)
Why we think this matters to the people who make the models
Every result above makes an NVIDIA model better at being an NVIDIA model. The flagship does frontier-benchmark work its raw mode can't. The mid-size shows +23 points of measured headroom that composition unlocks under our harness — capability its frozen forward pass couldn't reach alone in this regime. The edge-class model gets a measurable reasoning lift for the cost of an afternoon. None of it competes with the base model — all of it compounds the investment already sunk into training it.
And the direction of travel is the part we care about most: the same brain-attach mechanism that works on datacenter Nemotrons is the one we run on-device — our strongest deployed result to date is a 27B model gaining +26.5 points on a laptop, at 4-bit, no cloud in the loop (Note 4 and Note 6 cover why that matters). Reasoning that upgrades at the edge is reasoning that doesn't queue for a datacenter — which, as AI's share of grid demand keeps climbing, is a sentence we suspect resonates well beyond our lab.
The industry's default answer to "how do we get more capability" is "train a bigger model." These three results are the other answer, measured at 550B, 120B, and 3B-active on one family in one week: compose what's already trained. It's cheaper, it's faster, and it worked at every size we measured — on models already shipped.
And this is deliberately a beginning, not a conclusion. The attach protocol is repeatable and model-agnostic — dense, mixture-of-experts, and hybrid state-space bases measured so far — and the ladder we just climbed points one rung higher: the same brain, on the 550B flagship itself, is the next experiment in our queue. Pick a frozen model; the protocol runs.
Methodology: all figures are from real, logged runs; every claim traces to an entry in the campaign audit ledger. The 550B comparison matches the per-attempt reasoning budget (64k generation tokens) in both arms; the earlier 0/30 figure is retired and explained above rather than quietly dropped. The 120B n=200 confirmations and n=500 harm rows completed July 14–15; the pre-registered verdict window is this week. Nano rows are n=200/n=500, method-matched, full-forward; the multiple-choice deltas are individually within noise and reported as a no-harm row with a positive trend. No result in this note is extrapolated beyond the configurations tested.