A technique census of Phase 1: what the mathematics is doing, what walls it hit, and what's still open

ARC White-Box Estimation Challenge 2026
#1

Phase 1 is closed, and it looks like those who are going to publish a public write-up have done so. Here I present a census of the estimation techniques this problem admits, with measured verdicts. This post is an attempt to organize that census by what the mathematics is doing — techniques first, authors as citations.

Two scoping notes up front. First, this maps the published field only, and that is a heavily selected sample: one write-up reports scanning 1,405 graded submissions and finding 864 scoring better than its own, nearly all undocumented (amalgonim’s appendix, 18151) — score prizes reward silence, contribution prizes reward publishing, and the field split accordingly. Even the write-ups withhold components (jamesrahenry’s excludes a sampling-family estimator during the competition; konstantin_baltsat’s deliberately omits his public rank), so read each as a floor on what its authors know, and this census as a floor on what the field knows. Second, every claim below carries its source topic; corrections from the authors are welcome and half the point of posting.

The ARC problem. Given the weights of a random He-initialized ReLU MLP (width 256, depth 32, no biases), estimate the per-neuron final-layer mean E[h(X)] under X ~ N(0, I), scored by MSE against a Monte-Carlo reference, under a FLOP budget with dtype-aware pricing and a wall-time term. Two structural facts shape everything: ReLU’s positive homogeneity (radius and direction separate exactly), and the He-critical initialization (the network sits at the edge of chaos by construction).

The census at a glance.

  1. Parametric distribution propagation (moment/cumulant closures) — closed as a competitor: floor ~8.6–9.3e-7, mechanism understood from five directions (1a–1g)

  2. Monte Carlo + variance reduction — owns the published frontier; each gain priced (2a–2e)

  3. Deterministic spherical cubature — the angular problem effectively solved (Kerdock 5-design, two independent builds) (3a–3d)

  4. Structure exploitation — dead/on/kink routing; best published raw MSE (4a–4d)

  5. Control variates & blending — the most actionable open direction, priced at 3–5× (5a–5h)

  6. Metering, score, and benchmark mathematics — the instrument facts everything else depends on (6a–6h)

  7. Learned corrections — work as trajectory-fitted system components, die as local patches (7a–7c)

Then: convergences (C1–C5), the open mathematics (O1–O5), and what the census can’t see.

Technique 1 — Parametric distribution propagation (moment/cumulant closures).

The math: push a parametric family (Gaussian, or Gaussian + low cumulants) through affine maps and the ReLU nonlinearity, layer by layer, using closed-form rectified-Gaussian moments. The measured verdict: the family is closed as a competitor, with the mechanism now understood from five directions.

  • 1a. The two-moment readout has a floor at ~8.6–9.3e-7 — budget-independent, and in raw MSE ~40× worse than plain MC on identical draws (amalgonim, 18151). Error localization: the propagated mean carries essentially all of it (true mean + propagated variance scores 1.37e-6; the reverse, 7.07e-5).

  • 1b. Why the mean drifts: mean propagation is a unit-gain feedback loop — ∂M₁/∂μ ≈ 1 at criticality, so per-layer closure error is neither damped nor amplified, just carried and accumulated across all 32 layers (hyojun_kwon, 18154).

  • 1c. Why corrections inside the chain backfire: the chain is an error-compensating dynamical system — layerwise errors anticorrelate and partially cancel, so zero-mean perturbations damp while coherent injected corrections amplify at a measured ~16:1 (jamesrahenry, 18097; independently confirmed and credited in amalgonim’s appendix, 18151).

  • 1d. The k=2 closure’s dominant correctable error is a single multiplicative scale bias: optimal factor 0.9921 ± 0.0001 cross-validated, one scalar multiply, ~3× MSE reduction, and it matches full third-cumulant propagation at 1/500th the cost at this depth (pscamillo, 18063). No closed-form derivation of the 0.992 saturation exists — named there as an open problem.

  • 1e. Higher orders don’t rescue it: the organizers’ own k=3 reference collapses from 46× gain at depth 4 to 1.3× (and unstable) at depth 32, with error growing at an increasing rate per depth doubling (3.1× → 7.9× → 11.8×); k=4 exhausts 61.5 GB at depth 16 and SIGKILLs 122 GB at depth 32 (kwon, 18154). Exact-moment readouts bottom out at 1.47e-8 at order 4, then hurt — an asymptotic series (amalgonim appendix, 18151).

  • 1f. One important correction to the field’s folklore: several “Edgeworth makes it worse” readout numbers trace to a sign error in the standard derivation (the third-cumulant term of the rectified mean is −(g₁/6)aφ(a), not +); with the correct sign and true cumulants, the readout improves ~10× (8.85e-8), while in-chain injection still degrades for the compensation-dynamics reason above (amalgonim appendix, 18151, crediting jamesrahenry’s 18097). Two distinct effects, previously conflated — earlier exact-moment readout negatives (e.g. pscamillo’s, 18063) may be worth re-checking against the sign.

  • 1g. Certificate-grade version of the same wall: an exact full-covariance Gaussian recurrence, enclosure-certified per call, still sits 340× short of budget-matched sampling (jonah_butterbaugh, 18147).

Technique 2 — Monte Carlo and its variance-reduction mathematics.

The math: unbiased sampling, improved by structure — antithetic pairing (annihilates all odd harmonics, which also forces any control variate to be an even function — amalgonim appendix, 18151), randomized quasi-Monte-Carlo (randomly-shifted rank-1 lattices with a tent/baker fold; scrambled Sobol), input whitening, and Rao-Blackwellization of the exact first layer via homogeneity. The measured verdict: this family owns the honest frontier, and each gain has a measured price (amalgonim, 18151):

  • 2a. float32 casting: 2.00× (mandatory — float64 bills double)

  • 2b. randomly-shifted lattice + tent fold: 1.73×

  • 2c. input whitening: 1.05×

  • 2d. exact first layer (homogeneity identity): 1.055×

  • 2e. antithetic pairing alone: 1.007–1.05× (an earlier claim that antithetic loses was retracted as a measurement bug — equal-cost antithetic is bounded below at 0.5×, and measured ρ ≈ −0.03 to −0.05 makes it a small gain)

An unbiasedness-preserving frontier claim from the RQMC side: beating the lattice materially “seems to require a biased learned corrector” (evaaaz, 18053). A useful score identity: above the multiplier floor, adjusted score ≈ per-sample variance × per-sample cost — sample count cancels; only the product moves (amalgonim, 18151, item 5).

Technique 3 — Deterministic spherical cubature.

The math: homogeneity makes the radial integral exact in closed form, leaving a finite angular average; replace random directions with a spherical 5-design — 66,048 directions from 128 Kerdock/mutually-unbiased bases plus the coordinate basis, antipodal — certified by a fourth-moment identity, and evaluated at the first layer by signed fast Walsh–Hadamard transforms (eight butterfly stages) instead of a matmul. The measured verdict: for practical purposes the angular problem is solved, with two measured limits.

  • 3a. Two teams converged on the identical construction independently: graded raw 2.4258e-7 / adjusted 1.4391e-7 (konstantin_baltsat, 18149) and graded adjusted 1.55e-7 (skye_nygaard, 18145). A third stack reports the same family of banks (Cipo, 18152).

  • 3b. A basis-count ablation shows MSE ~ k^(−1.2) in the number of bases — the exponent exceeds 1, so under an adjusted score you never subsample the design (nygaard, 18145). A separate near-optimality claim rests on a computer-assisted certificate its own author quarantines as unreviewed.

  • 3c. The edge over the structure-aware sampler (Technique 4) is compute, not raw accuracy — its raw MSE is ~11% worse; the win is a cheaper carrier (baltsat, 18149).

  • 3d. A 5-design is still a degree-5 moment matcher, and one oracle analysis argues that alone cannot reach the leaderboard front (amalgonim appendix, 18151).

Technique 4 — Structure exploitation: classification, routing, sparsity.

The math: ReLU is piecewise linear, so a cheap diagonal-Gaussian pass over α = μ/σ classifies neurons as dead (α < −3: output exactly zero), on (α > 3: locally linear — propagate exactly), or kink (uncertain: spend the samples there), with pilot samples refining boundaries. Downstream, dead coordinates compact out of the matmuls, saturated regions linearize, and the remaining dense blocks take Strassen–Winograd (1.95× fewer multiplies) or row-bucketing. Measured: graded raw 2.18e-7 / adjusted 1.551e-7 — the best published raw MSE on the board (Team SOX: natasha_stewart and yangxinyu_xie, 18106, submission 319341; cross-checked by baltsat’s comparison arithmetic, 18149). The anatomy findings this technique surfaced:

  • 4a. The always-on neurons — the “easy” ones — carry 74.4% of final-layer squared error, as pure sampling variance (SOX, 18106).

  • 4b. Near-threshold neurons cross the kink together (cross-correlations growing with depth) — a joint event that independent-marginal propagation cannot represent (SOX, 18106).

  • 4c. From the closure side: ~97.5% of closure error concentrates in the ~66 neurons within 2σ of the kink; the costly non-Gaussianity is created by the first rectifier and never grows (amalgonim appendix §14.9, 18151).

  • 4d. 65% of the pair-block error is a lookup on (c_i, c_j, ρ) that saturates at 12 cells per axis and is identical across networks — a universal correction table may be buildable (amalgonim appendix, 18151).

Technique 5 — Control variates and analytic/sampler blending.

The math: report X − β(Y − E[Y]) where Y correlates with the sampler’s fluctuations and E[Y] is known analytically; variance falls by 1 − ρ². The census across write-ups yields a control-depth curve no single team measured:

  • 5a. layer-1 exact controls: ≈0.93× — a loss; shallow controls decorrelate (amalgonim appendix §7.20, 18151)

  • 5b. generic low-order controls: 1.0–1.2× (evaaaz, 18053; pscamillo, 18063)

  • 5c. inter-layer fluctuation-map controls: ~2× (Cipo, 18152)

  • 5d. mid-network activation control at layer 6, analytic propagation supplying its mean: 2.39× held-out (kwon, 18154). The same team’s oracle measurement prices a perfect penultimate control at 110×.

The design rules, all measured:

  • 5e. Deployable variance reduction is bounded by the analytic mean’s fidelity — “a bias problem wearing a variance-reduction costume” (kwon, 18154).

  • 5f. Under antithetic pairing, controls must be even functions — any linear control variate is identically zero (amalgonim appendix, 18151).

  • 5g. The control target must be deterministic: moments measured on the same rows are correlated with the error they’d correct, closing that route exactly (amalgonim appendix, 18151).

  • 5h. Per-MLP adaptive levers (control-layer choice, adaptive n, per-MLP fusion weights) are measured transfer traps — in-sample wins evaporate or invert held-out (kwon, 18154).

The endpoint of this spectrum — substituting the analytic close for the sampler at the final layer outright — was also built and graded: RQMC through 31 layers with a closure + Edgeworth + ridge close at the end scored 4.98e-7 after regrade, behind the plain unbiased samplers (radiant-allomancer, 18085); measured inside another participant’s sampler, the analytic last-layer close was 1.35× worse than the plain sample mean (jamesrahenry, 18097). The optimum is a blend, not a substitution.

The valuation that makes this line actionable: a mean-propagator faithful to b² ≈ 1e-6, blended with a frontier sampler, is priced at 3–5× on the score (“the largest number anywhere in this write-up”), and the deterministic pair-map chain needed to attempt it runs at 0.79% of budget. Accuracy, not cost, is the open problem (amalgonim appendix, 18151).

Technique 6 — Metering, score, and benchmark mathematics.

The score is part of the problem statement, and a sub-community measured it to instrument grade (amalgonim, 18151, with prior art credited to mliston 18108, ndrew1337 18101, brayden_siew 18092/18093, arianvassili 18105, and nkosi_ndwandwe 18127):

  • 6a. Residual wall time is a real compute channel: ~0.6M FLOP-equivalents per call on the grading stack (1–6M on typical hardware). Views/slices are free.

  • 6b. Concatenate carries residue 2.7× its own billed cost; fnp.block beats nested concat.

  • 6c. Row-bucketing beats Strassen once residuals count — and they are substitutes, worse combined.

  • 6d. Dtype pricing is 2×/1×/1× for f64/f32/f16 — f16 buys nothing and runs 16× slower on the backend, uncharged.

  • 6e. flopscope.stats silently promotes float32 to float64 (nkosi_ndwandwe, 18127); one three-operand einsum cannot run in float32 at all — the meter’s symmetry validation fails after 19 layers of cancellation (kwon, 18154).

  • 6f. Exact compiler-style rewrites with bitwise-identical outputs bought +3.1% adjusted in one graded pair (baltsat, 18149).

  • 6g. The ground truth was reverse-engineered: layer-0 closed forms let two teams infer the label set is a baked N ≈ 1e9 Monte-Carlo (radiant-allomancer, 18085; amalgonim’s appendix agrees on N but reports a label noise floor of ~6.1e-11 at the scored layer versus radiant-allomancer’s ~2e-10 — a public discrepancy the two might resolve between them; picked back up as O5 below).

  • 6h. The benchmark’s population was measured too: the 50 leaderboard MLPs are very unlikely to be a plain random draw of the full split — 0.19% of 20,000 bootstrap resamples land at or below the observed leaderboard-set mean — and the excluded upper tail carries ~36% of full-split error (keenanpepper, 18141). Whatever distribution the private re-evaluation draws from, the number the public board shows is not the number estimators will face.

Public ranks moved substantially in the endgame while regrades under v0.10.0 and a wave of dtype-aware resubmissions were both in flight, so read the final public standings loosely (baltsat, 18149, reports dropping 16→40 while improving).

Technique 7 — Learned corrections.

The math: fit a parametric corrector — ridge or linear heads on features the estimator already computes — against reference targets, then deploy it frozen. More teams touched this than any technique above, and the verdict splits cleanly on how the fit is done:

  • 7a. Fitted at scale on the corrected system’s own rolled-forward trajectory, it works as a system component: a per-layer stabilizer over the closure chain, ~8.4× over its baseline (jamesrahenry, 18097); an amortized per-layer ridge on the propagated mean (kwon, 18154); an offline ridge close on a sampler (radiant-allomancer, 18085).

  • 7b. Bolted on locally, it dies at validation: learned sign/scale models and weight features with no complete-score value (nygaard, 18145); local nonlinear residual heads 3.5× worse off-validation, alpha-tail ridge failing at 1.92×, global alpha calibration non-promotable (baltsat, 18149, Table 4); a learned bias-corrector reaching 2.3× where ~17× is needed (evaaaz, 18053); leave-networks-out gradient boosting on the final layer slightly worsening the residual, with a shuffled-label audit showing no generalizable signal (pscamillo, 18063).

  • 7c. The governing rules appear elsewhere in this census: fit on the corrected system’s own trajectory — 1c’s compensation dynamics is why anything else diverges — and treat per-MLP adaptivity as a transfer trap until proven otherwise (5h). Predictability itself decays with depth (τ ≈ 5.1 layers; pscamillo, 18063): whatever a cheap corrector can learn is gone by the scored layer unless the correction participates in the propagation.

The measured verdict: a trajectory-fitted corrector is a working component; a local patch is a validation casualty.

The convergences, compressed.

  • C1. The same closure wall was hit and measured five independent ways (pscamillo 18063, jamesrahenry 18097, butterbaugh 18147, amalgonim 18151, kwon 18154).

  • C2. Two teams built the identical 66,048-direction Kerdock 5-design (nygaard 18145, baltsat 18149).

  • C3. Four teams independently reassigned the analytic chain from predictor to infrastructure, into three jobs: router (SOX, 18106), control variate (Cipo, 18152; kwon, 18154), subtraction control/null — “effective as a control for subtraction but fails as a direct predictor” (butterbaugh, 18147).

  • C4. Three teams’ ground-truth sections run on the same public datasets of baked higher moments (jamesrahenry’s locked test set, 18097; pscamillo’s exact-moment tests, 18063; and a third team’s suite per its PDF) — which makes those datasets, published mid-competition by keenanpepper, the most load-bearing artifact in the census.

  • C5. The error-compensation account of why chained closures resist correction (jamesrahenry, 18097) was independently adopted by others’ falsification logs (amalgonim appendix, 18151).

The open mathematics.

  • O1. A closed-form derivation of the 0.992 mean-drift saturation from the propagation recurrence (named open by pscamillo, 18063).

  • O2. A mechanism theorem for the anticorrelated error dynamics — the unit-gain result (kwon, 18154) covers the mean channel; the covariance-channel compensation (jamesrahenry’s 16:1, 18097) has no derivation.

  • O3. The joint law of near-threshold neurons — multivariate probit is suggested by SOX (18106); the (c_i, c_j, ρ) universality in amalgonim’s appendix (18151) suggests it’s tabulable.

  • O4. Satisfying the oracle-derived necessary conditions for any deterministic method at the leaderboard front: layer-30 mean already at target accuracy, a non-moment readout, and a carried joint object of dimension ≥64 (amalgonim appendix, 18151) — no published technique satisfies all three.

  • O5. The label-floor discrepancy (6.1e-11 vs 2e-10) that bounds how far this benchmark can honestly be pushed.

What the census cannot see. The leaderboard’s front (raw MSE in the low 1e-9s) published nothing, and the strongest public forensics (amalgonim appendix, 18151) rule out moment propagation, low-rank quadrature, plain sampling, and CV/learned-corrector constructions, while showing public-half memorization is arithmetically insufficient. At that accuracy an estimate is within a few × of the baked ground truth’s own noise floor — the regime where estimating true means and matching the grader’s particular Monte-Carlo draw begin to blur. Whether the front is a technique missing from this census or something the fresh-MLP private re-evaluation will reframe is the most interesting open question Phase 1 leaves behind.

Method note. Corpus: all 53 topics in the challenge’s forum category as of 2026-08-13, enumerated with a per-topic attachment scan. Primary sources: the seven write-up PDFs attached in-forum, read in full (including one 115-page appendix); forum post text where no PDF exists. One write-up (topic 18147) is known to exist as a fuller PDF sent only to the organizers and is represented here by its forum abstract — the census’s one known coverage gap. The digestion was LLM-assisted and human-directed, like much of the literature it reviews.