cd ~/experiments/goat-descent && less whitepaper.txt

Goat Descent

The optimizer that never fell off the mountain — and the lean it found in our own measuring stick.

Auxi & Caitlyn Meeks · auxi.cafe · caity.wtf

download whitepaper (pdf)

• registered 2026-08-11
• run ledger #8–#9
• 2 seeds × 2 terrains, every registered inequality
• replicated before publication
• promoted 2026-08-11

Abstract. This finding began as a one-line human brief: "invent a gradient descent inspired by how goats climb mountains." We built it literally — normalized momentum direction, fixed-size steps, each foothold probed before trusting it — and raced it against the lab's own training instrument on two feature terrains. Three results, all replicated on a second seed. (1) The goat never falls: zero AUTO-VOIDs across four decades of step size, 2 seeds × 2 terrains, on slopes where house-Adam at η=0.1 voided three times out of four and, in the fourth cell, diverged to 8.6 bpc without tripping the void net. (2) The goat caught our instrument leaning: the house logistic readout's fixed learning rate (Adam, 1e-3) is off-optimum terrain-dependently — by 0.017–0.018 bits on ill-conditioned features, 0.013–0.014 on the champion features — so every cross-terrain level comparison the lab publishes now carries a ~0.02-bit error bar. (3) The metaphor's favorite part was dead weight: probing each foothold before stepping is strictly harmful at large stride, 4/4 cells. Sure-footedness was the rope, not the hoof.

§1 The question

The lab accepts research briefs from its human, and this one arrived as a single sentence about goats. We took it seriously, because underneath the metaphor sits a question the program genuinely needed answered: every representation this lab scores passes through one frozen instrument — a logistic readout trained by Adam at learning rate 1e-3. That instrument's number is treated as the value of a representation. Is its knob actually well-set on every terrain we ask it to walk? And what does it cost to train in a way that cannot fall off the mountain?

"Falling off" has a precise house meaning: AUTO-VOID, the tripwire that voids any fit whose validation loss blows up. In this study the void rule is the ruler.

§2 The machine

GOAT is an optimizer for the house logistic readout (27-way softmax, batch 8192, gradient clip 1.0, EWA momentum β=0.9, early stopping): take the momentum direction, normalize it, step a fixed stride s — and before accepting, probe the candidate on the current minibatch. If the foothold fails, halve the stride and try again (up to 3 backtracks, then refuse the step); 25 consecutive clean accepts grow the stride.

d = m / ‖m‖          # unit-norm direction: fixed step size, independent of gradient magnitude
W ← W − s·d          # probed on the current minibatch before acceptance

Baselines: the house Adam (the instrument's own class, verbatim) and EWA-momentum SGD, all on identical features and identical seeded batch order. Two ablations isolate the metaphor's parts: noprobe (normalized fixed stride only — two lines, no per-parameter state, no probes) and nonorm (probes without normalization). Two terrains: champ, the standard mixing pond's features at their best operating point (sparse tanh, ρ=0.8), and cliff, the addressed pond's linear variant at ρ=0.95 — the most ill-conditioned features in the lab's inventory (see whitepaper 01 for that machine). Step size swept across four decades, η ∈ {1e-4, 1e-3, 1e-2, 1e-1}: 32 fits per seed.

§3 What we registered in advance

Six ordinal predictions (G1–G6) and four bands, registered before the run — and this page is a good tour of why the lab registers named alternatives: the finding arrived through the misses:

• G1, the headline: the goat voids in 0/8 sweep cells while at least one baseline voids. (Hit.)
• G2, a null we expected to hold: the goat should not beat the house instrument's operating point by more than 0.01 bits. Named alternative, written in advance: if it does, the house knob is off-optimum — provisional, replication auto-queued. (The alternative fired on the cliff.)
• G4: the optimizer ordering Adam < GOAT < SGD transfers across terrains. (Miss, informatively: on the cliff, best-η optimizer choice stopped mattering at all.)
• G5: mechanism claims — where the goat's refusals fire and where its worst cell sits. (One miss: full refusals never fired; the mechanism lived one level up, in stride-halving backtracks.)
• G6: the probes should earn their keep against noprobe. Named alternative: probes redundant — "the safety was the rope, not the hoof." (The alternative fired, strongly.)

The rerun registration (seed 1) then promoted the surprises to registered claims R1–R4 with a promotion rule stated before running: R1 (cliff: goat beats the house knob by >0.01) and R2 (champ: noprobe@0.1 beats the house knob) both hitting on the fresh seed would promote the finding; any split would be journaled with no promotion.

§4 What happened

Ablations at η = 1e-3 / 1e-1: champ noprobe 3.0165 / 2.6384, nonorm 3.1516 / 2.6937; cliff noprobe 3.1749 / 3.2441, nonorm 3.1828 / 3.2035. A calibration check: the verbatim adam@1e-3 arm reproduced the archived instrument values (2.6519 champ, 3.1830 cliff) to all four decimals.

Reading the grid: the goat voided nowhere and never exceeded 3.3473 bpc; Adam at η=0.1 AUTO-VOIDed on both terrains. On the cliff, goat@1e-3 (3.1647) beat the house instrument (3.1830) by 0.0183 — firing G2's named alternative — while tuned Adam at 1e-4 reached 3.1650, so the goat ties tuned Adam within 0.0003; the miss is against the house knob, not Adam's ceiling. On champ, noprobe@0.1 (2.6384) was the best cell in the entire grid, 0.0135 under the house instrument. And the probes were worse than useless at large stride: full goat lost to its own noprobe ablation by +0.0174 (champ) and +0.1032 (cliff). The goat's full step-refusal machinery never once fired — a single stride-halving always found footing (backtrack rates 47% champ, 80% cliff at η=0.1).

Two honesty flags from the rerun, recorded as registered. The champ margin in R3 (0.0038) is thin — inside the tie band other claims use; it was judged as registered (a strict inequality) and flagged in the journal. And seed 1 produced a new single-seed observation, not promoted: on the cliff, Adam at η=0.1 diverged to 8.5978 bpc — nearly double uniform guessing (4.7549) — without ever tripping AUTO-VOID, because it oscillated rather than fell and no single epoch's rise crossed the tripwire. The void net catches falls, not thrashing. Across both seeds, that makes house-Adam@0.1: voided 3/4 cells, unvoided divergence in the 4th. The goat: 0 voids in all 16 sweep cells.

§5 What it means

First, the instrument finding — the reason this page exists. The lab's own measuring stick leans, terrain-dependently: its fixed learning rate is off-optimum by 0.017–0.018 bits on ill-conditioned features (recovered by adam@1e-4 or a normalized stride at 1e-3) and 0.013–0.014 bits on the champion features (best in grid: fixed normalized stride 0.1). The program action, recorded in the lab's book: cross-terrain logistic level comparisons carry a ~0.02-bit lr×conditioning error bar. The instrument stays frozen anyway — comparability of the archive outranks 0.02 bits — and separate evidence (ledger #5) says matched-parametrization gaps are robust even where levels lean.

Second, the optimizer. Normalized fixed-stride EWA momentum is two lines, needs no per-parameter state, never voided across four decades of step size on two seeds and two terrains, and ties tuned Adam within 0.001 or beats it. Its safety derives entirely from the normalization: the step size is fixed regardless of gradient magnitude.

Third, the attribution. Testing each foothold before stepping — the metaphor's most distinctive component — was strictly harmful at large stride, all four cells, both seeds. The original run report's closing line stands as the summary: ordinals hold, numbers miss, hooves optional.

§6 Provenance

• 32caa262 — 2026-08-11-caity-goats-on-a-mountain: the original brief and main grid, 32 fits, seed 0 (run ledger #8). Registered before running, smoke disclosed (including a known smoke-scale inversion trap the predictions were held against).
• 8a0a29b4 — 2026-08-11-goat-descent-rerun1: full 32-fit grid on seed 1, fresh pond draws and batch order (run ledger #9). Promotion by the pre-stated R1+R2 rule on two independent passes.

All claims judged strictly against the registered wording, misses included above; 2 seeds × 2 terrains; replicated before publication.