W4M · Research Notes

Research Note · Racing · 18 / 19

Stop paying for thinking a problem doesn't need

Note 15 showed that a frozen flagship — NVIDIA's Nemotron-3-Ultra-550B-A55B — wrapped in a verify-and-select loop, fixes bugs it couldn't fix alone. This note asks the follow-up an operator would ask: how much of that sampling do you actually need? We froze an adaptive stopping policy before a single token was generated, ran it live on 200 held-out problems, and let it decide — per problem — when to stop sampling. It beat the fixed-budget reference on quality while leaving a third of the budget unspent. One of its two pre-registered bars missed; that miss is in this note too, at full size, with the measured reason.

W4M Research · July 2026 · ~5 min read

129 / 200
Live, pre-registered adaptive run on SWE-bench Verified: above the frozen quality bar (117) and above the fixed-budget reference (119) — decided prospectively, not in replay.
33% saved
of the fixed sampling budget, un-spent. The policy stopped early on 89 problems — 73 of them after a single sample — and never broke a solved one.
1 of 2
pre-registered bars missed: average samples came in at 3.345 vs the 3.25 bar. Published anyway, root-caused, and the fix is frozen into the next pre-registration.
The short version Sampling a model several times and keeping the best verified answer works — that was Note 15's +7-point, p = 1.9×10⁻⁹ result on the frozen 550B-class Nemotron flagship. But fixed k-times sampling is wasteful: most problems don't need five attempts. So we froze a stopping rule in advance — the policy watches each sample's execution-verified quality and decides when more sampling won't pay — and ran it live, on 200 problems it had never touched, with two bars registered before any generation: match the fixed-budget reference's quality, and do it under a set budget. Quality: passed, and then some — 129 resolved vs the reference's 119. Budget: missed by a hair (3.345 average samples vs 3.25), because a rule calibrated on replayed data stops slightly less often on fresh samples. Both outcomes are in this note because both were promised in advance.

The posture note that opens every note in this series: the numbers below are from real, logged runs; the policy, the bars, and the problem list were sha-frozen before any generation; and negative or missed results appear at the same size as wins. The benchmark is public (SWE-bench Verified); the receipts are reproducible under NDA.

What ran, exactly

Two hundred problems, drawn by seed from the same paired set as Note 15's definitive run. For each problem the frozen flagship generates one candidate fix at a time. After each candidate, an execution-based check — the same non-leaking public-test signal the selection loop already uses — scores it, and the frozen policy either stops (the fix looks done) or samples again, up to five attempts. No human in the loop, no threshold tuned after the fact, no problem excluded.

Pre-registered bar (frozen before generation)ResultVerdict
Quality: resolve ≥ 117/200 (within 1 point of the fixed-budget reference, 119)129 / 200 — above the bar and the referencepassed
Budget: ≤ 3.25 samples per problem on average3.345missed

The anatomy is the interesting part. 73 problems closed on their very first sample — the policy recognized a verified fix immediately and spent nothing more. Eleven more closed on the second, then the tail thinned fast (2, 2, 1). The remaining 111 ran to the cap and kept their best-scoring attempt. Net: the full fixed-budget quality, and then three more points of it, with 33% of the sampling spend saved.

The policy never broke a solved problem — same property as the selection loop it sits on. Racing is how the +7-point result becomes affordable, not a new way to risk it.

The miss, at full size

The budget bar said 3.25 average samples; the live run spent 3.345. We did the autopsy on the per-sample records rather than accepting our own first explanation — and the first explanation was wrong. The live run's first wave actually matched the replay almost exactly. The real cost hid in two places. First, seventeen problems had a fully passing fix on the very first sample but the independent reproduction step — the extra check the policy requires before it stops — read as failed; a simulation said a single retry on that step would land the run at ~2.95 average samples, comfortably under the bar. Second, the stopping rule's threshold sits in a gap of the score distribution, so the obvious fix (re-tune the threshold) provably changes nothing — the better rule lowers the bar to stop as the remaining budget shrinks, the classic shape from optimal stopping.

Then we measured the first fix instead of trusting its simulation — and the simulation was optimistic. Re-running the reproduction step for every candidate that had lost a close this way recovered only 6 of 74. Most of those failures were the check working, not infrastructure flaking; the retry alone would not have carried the run under the bar. So the correction gets the same size as the claim: the decaying stop-threshold is the fix that goes forward, the retry is a minor assist, and both the original simulation and the measurement that demoted it are in the logged records. We could have shipped this note without the miss — and without the correction of our own autopsy. It would have been a different company's note.

Update — the redemption run (Aug 2) The fix went forward exactly as promised, preregistered before the first token: the reproduction-step retry plus the wave-decaying stop threshold (140→130→120→110), frozen bars unchanged. The re-run's verdict: quality 125/200 — PASS (bar 117) and budget 2.880 average samples — PASS (bar 3.25), against the pilot's 129 / 3.345 miss. Quality gave up four resolutions and stayed above the bar; spend dropped 14%. One more thing the re-run taught us that the simulation didn't: the final wave was never needed — both bars passed with a wave still in hand, and the un-run wave is the savings. That is the product behavior, measured on itself.

Why this generalizes — early signal from three other model families

The headline generalization first: on a current-generation open model — Google's Gemma-4 26B-A4B — deciding per problem whether to deliberate at all cut reasoning-token spend by 79% — four out of five reasoning tokens simply not spent — with accuracy inside the noise band. When this note was drafted that was a probe-scale figure; the full-set runs landed overnight and it held — same model, same sample count, the cheap mode matches the deliberate one on the full 500-problem MATH set at a fifth of the tokens. The overnight runs also produced the sharpest row yet: on an easier benchmark — GSM8K — the same model is 2.6 points more accurate while spending 86% fewer reasoning tokens — skipping deliberation there isn't a trade, it's a strict win. And the dividend is lawful across the family at matched sample count: deliberation pays clearly on the smallest model (about +21 points under artifact-corrected grading, 61.6 → 82.2 with the same corrected grader on both arms), and nothing at all on the tiers above it — which is exactly why the decision has to be made per problem, not per model. On Alibaba's Qwen3.6-35B-A3B, a frontier open mixture-of-experts model, an agreement-based race matched full-consensus accuracy with 31% fewer reasoning tokens than full-consensus voting (probe-scale, full run scheduled) — a cheaper way to buy the multi-sample quality lift, still roughly 3× a single answer's cost, stated plainly. And on a small edge-class model — Google's Gemma-3 1B — verified selection with a larger in-family judge (Gemma-3 12B) lifted grade-school math (GSM8K) from 60% to just under 80 — most of the gap to a model four times its size — with no training at all.

The deepest of the overnight cuts deserves its own paragraph. Give the deliberating edge model — Gemma-4 E4B — and the dense flagship — Gemma-4 31Btwenty samples each on the full competition-math set (MATH) and majority voting calls it a dead heat — 85.0 against 85.2. But the samples themselves disagree: for 91.6% of the problems, at least one of the edge model's twenty samples is correct — for the flagship, 88.2% — and the flagship's ceiling had already flattened while the edge's was still climbing. Voting cannot see that difference. Only a selector that can verify which sample is right can harvest it — and that 3.4-point gap, invisible to consensus, is exactly the machine this series of notes is about.

Identical accuracy, dramatically less spend — three routing axes, measured
everything saved (100%) Mode pair · 26B-A4B GSM8K full-set · accuracy 2.6pts HIGHER 86% fewer reasoning tokens Mode routing · 26B-A4B full-set · accuracy within noise 79% fewer reasoning tokens Adaptive sampling · Nemotron live pilot · quality above reference 33% less sampling budget Agreement race · Qwen3.6 identical accuracy 31% fewer reasoning tokens
Reasoning spend saved, per routing axis — the green fill is what the decide-per-problem policy did not have to spend, out of the naive spend (always-deliberate, full consensus, fixed k-sampling). The Nemotron row is this note's live pre-registered pilot; the two mode rows went full-set (n=500) confirmatory overnight; the Qwen3.6 race row remains probe-scale (full run scheduled). Every figure traces to a logged run.
Honest note The 129/200 headline is a live, prospective result under frozen rules — but it is one run of one policy at n=200, and one of its two bars missed. The mode-routing figures in the last section got their full-set treatment overnight (n=500 per cell, matched sample count, zero generation errors) and held; the Qwen3.6 race figure remains probe-scale and is labeled as such. Three programs also closed negative this cycle under their own frozen gates — two memory write-path variants and one small-model training pilot — and those washes are logged with the same rigor as this note's win. Nothing here touches model weights; nothing here reads held-out tests; every number traces to a logged artifact.
For the technical reader Design: n=200 seeded from the definitive paired set; per-problem sequential sampling from the frozen Nemotron-3-Ultra-550B-A55B at deployment precision, cap k=5; after each sample a non-leaking execution signal (public regression tests + validated reproduction only; held-out tests never read) scores the candidate; a stopping threshold frozen from an earlier replay decides continue/stop; final answer is the first stopping sample, else the best-scoring of five. Bars frozen before generation: resolve ≥117/200 (reference 119/200 from fixed k=5 on the same problems); mean samples ≤3.25. Outcome: 129/200 resolved (official harness grading), closes per wave 73/11/2/2/1, mean samples 3.345 (bar missed; autopsy on per-sample records: wave-1 close-rate matched replay within 1pp — the miss decomposes into (i) 17 wave-1 instances with full public-test passes lost at the reproduction-validation step (retry-on-invalid simulated to mean 2.945, but a full measured re-validation of all 77 lost-close candidates recovered only 6/74 — the step is a hard gate, the simulation was optimistic, correction published at the same prominence) and (ii) a stopping threshold sitting in a bimodal-score gap, making flat re-calibration a no-op — replaced by a wave-decaying schedule (simulates to 2.975); the decaying schedule carries the next pre-registration). Cross-family probes: reasoning-mode routing (fast-vs-deliberate) at accuracy parity and ≈21% of reasoning tokens on Gemma-4 26B-A4B; agreement-race at consensus parity and 69% of tokens on Qwen3.6-35B-A3B; k=20 judged selection +19.5 points on Gemma-3 1B with an in-family judge two size tiers up (Gemma-3 12B). Selection internals, thresholds, and the judging/gating architecture are proprietary; full pre-registration shas, per-instance records, and reproduction paths are available to qualified partners under NDA.

The series' claim keeps compounding: a frozen model, wrapped well, does more than it does alone — and now it does so while deciding, per problem, how much of your money to spend. Quality above the fixed-budget reference at two-thirds the budget, live, under rules we couldn't retro-fit. And when one of our own bars missed, it went in the note at full size — because the discipline is the product too.

Methodology: all figures from logged runs. The live pilot's policy, thresholds, problem list, and both bars were sha-frozen before any generation; grading used the benchmark's official harness; no problems were excluded and no thresholds were adjusted after freezing. Cross-family mode-routing figures are full-set (n=500) measurements at vendor-recommended inference configurations with standard symbolic-equivalence grading for math; the remaining probe-scale figures are labeled as such and scheduled for full-set paired runs. No model weights were modified anywhere in this note.

Models & sources

The flagship under test: NVIDIA's Nemotron-3-Ultra-550B-A55B (550B-class) (open weights, served frozen on our stack). Benchmark: SWE-bench Verified — the human-validated subset of SWE-bench — run through the open-source OpenHands agent harness at its standard configuration. The cross-family models probed at the end of this note: Google's Gemma-4 26B-A4B, Gemma-4 E4B, and Gemma-4 31B, plus Gemma-3 1B with Gemma-3 12B as judge; and Alibaba's Qwen3.6-35B-A3B. Math benchmarks: MATH (full sets, n=500) and GSM8K. These families get full-set treatment in Note 19.