GoM · One real training run — recorded, not live

It learned a rule nobody taught it.

Same task. Two systems. The plain language model keeps guessing — flat at 14.5%. GoM watches examples stream past, reasons step by step, checks its own work, and learns from feedback — climbing from chance to 73% on held-out items. Replayed here from one real training run — based on real data, not a live demo.

The task. A hidden rule of the form answer = color × position (mod M) decides the correct answer for every item — and neither system is ever shown the rule. One example is a single item presented for a guess; 960,000 of them stream past over the run, and the learner's only signal is feedback on each guess — it must infer the rule from that alone. Accuracy is scored on held-out examples it has never seen, and the flat 14.5% line is what chance-level guessing looks like: a frozen model that cannot learn stays exactly there, 960,000 examples later.

What this proves: in-session learning from feedback alone — the frozen model cannot move, and the learner climbs from chance toward the scaffold's ceiling.

Examples seen 0

GoM accuracy 4%

Bare LLM 14.5%

GoM — learns from feedback Belief scaffold — full reasoning trace Bare LLM — no learning, still guessing

Hidden rule · discovered

answer = color × position (mod M)

Never shown the rule. Inferred it from feedback alone.

What you're looking at: one real training run, recorded and replayed — not a live demo, and not idealized. The line spikes early because GoM memorizes a batch (overfits to 100% on a handful of items); that shortcut is detected and reset, and the model then re-learns the underlying rule, climbing to 73% on held-out examples. The 69% headline below is the average across the full n = 504 multi-seed campaign — a single run can land a little above or below it. The violet line is the upper bound — the same model handed the full reasoning scaffold; the learner's job is to reach it from feedback alone. Raw per-checkpoint values: learn_data.json (shipped with this page).

Bare LLM · before & after

14.5%

Still guessing. A frozen model never moves — 960,000 examples later, exactly where it started.

→

GoM · it learned it

69%

Taught itself the rule. From a 4% standing start to 69% — and it keeps the lesson on examples it had never seen.

The validated result

95% CI · n = 504–1,000

0.855 → 0.310

Error rate, GoM. From near-total failure to reliable.

0.855

Error rate of no-learning controls — unchanged.

0.017

Learned the rule, not the answers — held-out accuracy within 1.7 points of training accuracy.

0.0

Error with the full reasoning scaffold — perfect.

Read: across a multi-seed campaign (n = 504–1,000 held-out items, 95% confidence intervals), GoM cut error from 0.855 → 0.310 while every no-learning control stayed at 0.855. The 0.017 train-to-held-out gap means it discovered the underlying rule and applied it to brand-new examples — generalization, not memorization.

A model that improves in the field, without a retraining cycle.

It learned a rule nobody taught it.

The validated result

See the rest of the deck.