[Phase 2 write-up] A factorized K=3 cumulant chain whose corrections are fitted through its linearised mean propagation (submission #337048)

This is our technical write-up for the Phase 2 Algorithmic Contribution Prize. The attached PDF is the write-up we emailed to arc-whestbench@aicrowd.com on 21 October, with three pages added in front: a plain summary and a list of related public work. Everything from its title on is unchanged.

Submission ID: #337048 (public score 1.6138413e-09, all 100 MLPs completed). Our second nominated submission is #337054.

The short version:

  • The estimator is the organisers’ factorized K=3 cumulant chain, built on 504aldo’s public code (#330093). The table “What is ours” lists every part and whose it is; the young-tier star terms, the largest accuracy part in the file, build on marius_binner’s published description.
  • Our main tool is the linearised mean propagation P_t = diag(Phi(a_t)) W_t^T. Measured through P, each layer passes 0.600 to 0.872 of an inherited error’s squared norm, within 0.0142 of the infinite-width ReLU law 1/2 + arcsin(rho)/pi at every junction but layer 2. At layer 2 the gain is 0.702 against the law’s 0.664, because the layer-1 correction leaves part of its error along the mean vector, which P passes more strongly.
  • Since the final-layer error is linear in the corrections to first order through P, one ridge regression fits all 840 correction coefficients to the final-layer error. In two older builds, refitting the residual corrector’s readout to its own layers’ error made layers 1-5 6.4% better and the output no better.
  • Six accuracy parts of ours, each switched off alone in the final file, multiply the final-layer MSE by 1.01321 to 1.15954, each at least 6.0 standard errors above 1.
  • The score is about 3.3 times below #330093; cost carries 59% of that factor in log terms, billing-rule changes included. A guard ladder keeps every MLP under the wall and residual caps. On the grader the client’s live residual is unavailable during a predict, so the picks compute it themselves.
  • Negative results: a differentiable twin finds no step that is better on fresh networks, and Section 7 lists what did not work, with numbers.

Related public work that our Section 9 does not cover: megan_park (18282) states the survival law more clearly than we do, and dogus_ozel measured the same per-layer loss in their chain (18281); the law itself is classical (Cho and Saul 2009; Schoenholz et al. 2017). Artyom_Mazur (18249) treats errors as random projections, kai_leal_gonzalez (18246) gives a two-term closure for the (3,1) slice, and il.miscusi (18248) removes the global scale mode. yeongjunyoo (18284) and Susupro (18254) also carry corrections through the same mean derivative.

LLM use: the code, the experiments and the drafts of the text were produced by Claude (Anthropic) agents in sessions I directed; Section 10 gives the details.

64 pages: the 3-page overview, then the write-up (16 pages of main text, then Appendices A to I).

Phase2_writeup_337048.pdf (1.0 MB)

2 Likes