Phase 1 write-up: Stabilizing cumulant propagation at depth 32 — a trajectory-calibrated moment chain, with an error budget (submission #314695)

[jamesrahenry]

This is the summary of our algorithmic-contribution write-up; the full 16-page PDF is attached below, and every script and results file it references is public in the replication repository (MIT). Referenced graded submission: #314695 (raw 5.89e-6, multiplier 0.169, adjusted 9.97e-7).

Disclosure: this work was AI-assisted (Claude, Anthropic). The model proposed experiments, wrote the code, and drafted the documents; a human directed the research and reviewed the results. The write-up was independently reviewed twice against the underlying scripts and data; details in its Disclosure section.

The central finding. A deep moment-propagation chain behaves as an error-compensating dynamical system: its per-layer errors are anticorrelated and partially cancel, so the chained estimator attenuates zero-mean noise while amplifying small coherent biases (measured end-to-end at a ~16:1 exchange rate). This one property explains why step-exact Edgeworth corrections from true cumulants make the chained estimator worse, why corrections succeed only when fitted on the chain’s own rolled-forward trajectories (DAgger-style), why a variance-reducing shrinkage filter doubled the final error despite working as designed, and why layerwise bias re-anchoring failed the same way in radiant-allomancer’s independent experiments.

Contributions, briefly:

1. Diagnosis. The depth-32 failure of k=2 propagation is error dynamics, not closure order: with true input moments a single closure step is already below a strong sampling estimator’s total raw error (1.16e-6 vs 1.61e-6); chained, the same closure is 52× worse, and re-anchoring the covariance to truth makes it worse still.

2. Method. Per-layer linear corrections fitted on the chain’s own rolled-forward trajectories reach 7.4e-6 held-out at seed-resampled expectation, ~8.4× better than plain kprop at matched budget, graded as #314695.

3. Ground-truth joint-cumulant tests. Using keenanpepper’s published full K3 tensors (evaluation-only, pre-registered locked-set policy): the joint structure missing from the closure is real and compressible, yet an analytic carrier of it, at reconstruction cosine 0.83, still loses 4.5× to a plain N=4096 MC probe, 10/10 IDs.

4. An error budget for the graded artifact. Roughly half of #314695’s final error is probe-sampling noise, attributed by field (the pair field dominates, and its noise anticorrelates with κ3’s, so denoising κ3 alone makes things worse), with a measured denoising ceiling and three instrumented exploitation failures.

5. Negative results with mechanisms, and an open problem: a lower-variance unbiased estimator of {κ3, E[zc²ᵢzcⱼ]} at fixed sample count is worth up to ~2× raw MSE to any probe-fed correction scheme; shrinkage-family estimators are excluded by the bias-amplification dynamics, not by their variance performance.

On the town-hall hypothesis. We confirm the premise quantitatively: the pre-activation covariance participation ratio contracts 128 → 5.2 (25×) over the 32 layers. We then tested the natural implementation (a subspace-split hybrid) and found it negative with a structural mechanism: sampling noise concentrates in the same low-rank subspace as the signal (λᵢ/N, direction by direction), so the complement carries only ~1.7% of sampling error and even a perfect analytic side is capped at ~2%. A method exploiting the low rank must either beat λᵢ/N in the leading directions or convert rank into FLOP reduction. Details and the depth-graded rank budget are in §6 of the PDF.

Relation to prior write-ups. We read pscamillo’s and evaaaz’s write-ups and radiant-allomancer’s before finalizing ours; §6 of the PDF details where our results corroborate theirs (three of radiant-allomancer’s negatives match ours mechanism-for-mechanism, from the opposite architecture) and where we tested their components in our setting. Particular thanks to keenanpepper, whose two public datasets (arc-whestbench-higher-moments-2026, whest-k3-tensors-2026) made the fitting targets and the ground-truth tests possible at all.

Sampling wins Phase 1’s raw metric, and the write-up says so plainly; the analytic track targets the regimes where sampling itself is unreliable. We would welcome cross-checks — in particular of trajectory-fitting against pscamillo’s falsification suite, and of the probe-noise decomposition (which applies to any probe-fed estimator, including hybrids) against radiant-allomancer’s two-budget method.

Henry_ARC-WhiteBox_Phase1_Writeup_Submission-314695.pdf (119.9 KB)

[jamesrahenry]

Erratum: the FLOPScope v0.10.0 repricing (announced here) regraded the submissions this write-up cites. The science is unaffected; the grader-reported scores it quotes are superseded as follows.

Our write-up’s numbers were graded under the previous, dtype-blind cost model. Under v0.10.0 (float64 bills 2× float32, priced by output dtype), the official record now reads:

submission	quoted in the write-up	official after regrade
#314695 (the write-up’s artifact)	adjusted 9.97e-7, multiplier 0.169	adjusted 1.76842e-6, multiplier 0.300
our sampling entry	adjusted 3.17e-7	superseded by a float32 resubmission, #323492, adjusted 3.07e-7 (raw identical)
pscamillo, #314331	adjusted 2.45e-6	2.4501e-6 — unchanged (at the multiplier floor, insensitive to the repricing)
radiant-allomancer, #317660	adjusted 4.47e-7	4.98e-7

No raw MSE changed (the regrade left #314695’s raw byte-identical at 5.89e-6), so the diagnosis, the trajectory-fitted stabilization, the ground-truth cumulant tests, and the error budget all stand as written.

Why #314695 moved the most of these: its hot path runs in default float64 — the probe draws float64 normals, so every matmul against the grader’s float32 weights promotes to the 2× rate. Re-metered under flopscope 0.10.0 (repricing_erratum_check.py in the replication repository), the graded file bills 7.40e10 FLOPs per net vs 3.70e10 for a float32 port with identical arithmetic (ratio 1.999, output delta ≤ 2.7e-6 against a per-element RMSE of ~2.4e-3). A float32 resubmission would grade at multiplier ≈ 0.15, adjusted ≈ 0.9e-6 — essentially the originally-quoted numbers. We are not resubmitting it: the write-up’s claim is mechanistic, not positional.

Two corrections to the text itself. The wall-time paragraph (which attributed the 13.6% → 0.169 gap to Python-dispatch wall time) is superseded: under v0.10.0 the regraded multiplier is accounted for by the FLOP bill alone, and dtype — not wall-time batching — is now the dominant multiplier lever for this estimator family. And the §3 economics line should read multiplier ≈ 0.15–0.30, adjusted ≈ 0.9–1.8e-6 under the new model; the conclusion (well short of the sampling frontier) is unchanged, and in fact strengthened — the repricing hit dtype-naive mid-multiplier entries like ours hardest, while the frontier entries sit at the multiplier floor and barely moved.

One practical note for other teams, in the spirit of Mohanty’s post: if your estimator never sets a dtype, your entire hot path is float64 and your bill has roughly doubled. Casting activations to float32 after any tail-sensitive transforms recovers the old bill; in our sampling entry the inverse-CDF stays float64 precisely because Sobol uniforms can round to exactly 1.0 in float32 and NaN the tail branch.

Henry_ARC-WhiteBox_Phase1_Writeup_Submission-314695_erratum-v0100.pdf (125.6 KB)

[jamesrahenry]

Write-up addendum (2026-08-10): the stabilizer schedule is state, not depth.

Following the deadline extension for mechanistic contributions, I’ve added a second dated addendum to my Phase 1 write-up (submission #314695, PDF re-rendered; the graded-time text remains unchanged, same convention as the 2026-08-04 erratum). Everything in it post-dates both the submission deadline and my private re-evaluation selections, and none of it was used in any graded submission. Scripts and committed results are in a separate folder of the replication repo: arc-whitebox-replication/post_phase1_state_keyed at master · jamesrahenry/arc-whitebox-replication · GitHub

The short version. The graded estimator’s only depth-32 dependency is its stabilizer coefficient table: 512 doubles indexed by layer number. I refit the same 16 correction slots as a function of measured per-layer state — participation ratio of the propagated covariance (the rank-contraction profile from §6, now used as a control signal), the ReLU operating point (mean/std of μ/σ), and off-diagonal correlation mass — with no layer index anywhere and 128 parameters instead of 512. Three results:

1. Parity at depth 32. Held-out 4.74e-6 vs the deployed table’s 5.21e-6 (5/10 per-net wins; a validation-selection caveat on the exact figure is stated in the addendum). The claim is parity with 4× fewer parameters, not superiority.

2. Both schemes extrapolate to depths 48 and 64 — and the reason is the finding. I expected index-clamping past row 31 to fail out-of-range. It doesn’t, because the chain’s state trajectory saturates past roughly L30: rank contraction completes, drift slows, and the last row stays approximately valid forever. The index table survives because it was secretly a state lookup whose state stops changing. Consistent with this, depth-64 error is lower than depth-32 — the saturated regime is effectively low-dimensional. State-keying gets the same transfer with zero fitted data past layer 32, which is the cleaner statement: the schedule is a function of where a net is in its rank-collapse trajectory, not how deep it is.

3. On 200 unseen public-split nets, the improvement concentrates in the hard tail. Overall 6.92e-6 vs 7.60e-6 (125/200 wins), but split by difficulty the improvement is 1.06× in the body vs 1.20× in the p90+ tail, corr(log difficulty, log improvement) = +0.31. Mechanistic read: part of what makes the hard nets hard is not irreducible variance but correction-model mismatch on nets whose internal state at layer l is atypical for layer l. This was directly prompted by @keenanpepper’s public-50 distribution analysis — it suggests the upper tail carrying ~36% of full-split error is, in part, the atypical-state-trajectory population, and yields a falsifiable prediction I haven’t tested: tail membership should be predictable from weights alone, via distance of a net’s state-descriptor trajectory from the ensemble band. No MC needed. If anyone wants to check that against the 512-net split before I do, please be my guest.

One negative result for the methods record, consistent with §5 of the write-up: a pooled fit (all layers at once, iterated) diverged within two refit iterations; the sequential fit-on-the-chain’s-own-trajectory structure was stable throughout. The keying can change; the trajectory-fitting requirement cannot.

Added at posting time (2026-08-11), since several post-close write-ups landed after this addendum was rendered: @amalgonim’s write-up (submission 324409, topic 18151) independently finds that Edgeworth corrections degrade the chained closure (3.8×) — §1c of my write-up, replicated with different machinery — and measures the closure family’s floor at 8.8e-7, putting a number on §5’s “well short of the sampling frontier.” Meanwhile both @jonah_butterbaugh’s write-up (submission 326094, topic 18147: exact Gaussian structure is “effective as a control for subtraction but fails as a direct predictor”) and @Cipo’s post-mortem (topic 18152: control variates built on inter-layer correlations) arrive at the role §8 proposed for this machinery — the analytic chain’s frontier value is as a control/null model, not a predictor. The state-keyed line above is aimed at exactly that role: a null you can evaluate per-net, cheaply, from the weights alone.

Henry_ARC-WhiteBox_Phase1_Writeup_Submission-314695_addendum-20260810.pdf (148.5 KB)

[jamesrahenry]

Follow-up (2026-08-11):
We tested our own tail prediction, and half of it dies. The addendum predicted tail membership would be predictable from weights alone via distance of a net’s state-descriptor trajectory from the ensemble band. We ran that test twice — single-probe-seed on 200 nets, then the decisive version on 100 nets × 8 probe seeds with difficulty split into bias² and variance components. Results: the hard tail is real, not probe luck (single-seed vs 8-seed intrinsic difficulty, Spearman +0.82), and the mechanism claim strengthens — state-keyed corrections improve the intrinsic tail 1.32× vs 1.07× in the body. But the weights-only predictor fails: band-distance atypicality reaches only +0.24 against the bias² component and retrieves the tail at chance level.

Also tested: an ensemble version of the kink-neuron hypothesis (from Natasha Stewart’s write-up) — kink fraction correlates negatively with difficulty (−0.19), so “more boundary neurons → harder net” is wrong at net level too. A Davis–Kahan-motivated eigengap statistic (subspace sensitivity ∝ 1/gap, computed from deterministically propagated covariances) also comes back null — He-random nets at this width appear spectrally self-averaging, so the difficulty signature is not in any marginal spectral summary.

Our reconciliation: the state-keyed estimator helps the tail without ever needing to know which nets are hard — it reads state per-layer and corrects wherever the deviation is correction-relevant, while a symmetric average distance destroys exactly that direction- and depth-specific structure. So: mechanism confirmed, operationalization retracted. If someone finds a weights-only statistic that does retrieve the tail, we’d genuinely like to see it. Scripts and results committed alongside the rest: post_phase1_state_keyed/ (kink_fraction_tail_check.py, multiseed_tail_check.py).