Submission #336225: adjusted 3.8132e-9, final-layer MSE 2.0994e-8, q 0.1816. Full write-up (PDF) attached.
This is a month of work on top of 504aldo’s public V29 (MIT, forum topic 18218), the factorized K = 3 cumulant propagation of arXiv:2605.05179 with a memoryless fourth cumulant. V29 is a strong base: 5.7x cheaper than the organizers’ factorized reference at a better MSE. We extended it with an all-distinct kappa4 edge term (free through an exact D21 = M W^T identity), a kappa3-covariance edge history (graded MSE 1.9951e-8, #336215), two exact term fixes, a fitted (3,1) law and a cost engineering pass. The main result we want to share is measured: where this representation ends, and why, so that nobody has to spend a month finding it out again.
1. The error budget (oracle injections into the production chain)
Exact values replace one carried quantity at layers 8-15; the number is the final MSE ratio. “16M” is a net scored against its own 16M-path means, “pub0” is public net 0 against the organizers’ 1e9-path truth.
| exact at layers 8-15 | 16M | pub0 |
|---|---|---|
| var | 0.687 | 0.647 |
| C_off | 0.778 | 0.819 |
| D21 | 0.954 | |
| d3 + g4 | 0.610 | 0.508 |
| var + d3 + g4 | 0.190 (oracle floor) | 0.127 |
| (3,1) kappa4 slice, on the graded engine | 0.909 |
The final error sits in three per-neuron inputs of the mean map (var, d3, g4) of near-gate neurons at layers 8-15; the whole covariance side made exact leaves 70 % of it. d3 and g4 errors are the all-distinct kappa3 / kappa4 memory. The channels cancel: an exact (2,2) slice alone raises the MSE (1.033), and the LS-fitted (3,1) law works best scaled by 0.6-0.8.
2. What the memory cannot lose (measured)
| change to the kappa3 memory | MSE |
|---|---|
| all-distinct part set to zero (n = 256 / 512 / 1024) | 13x / 23x / 43x |
| Tucker rank 256 / 512 / 768 | 13.5x / 3.7x / 1.36x |
| drop sources older than 3 / 5 / 8 / 12 layers | 43x / 19x / 5.0x / 1.25x |
| old sources merged into one exactly transported slice plane (K = 4 / 3 / 2) | 7.2x / 11.7x / 17.8x |
| young A leg compressed to rank 256 (both slices) / 768 (D21 slice only) | 14.5x / 1.035x |
| old legs in one carried basis, rank 256 / 384 / 512 (rel. error of their slices at layer 15) | 2.4 % / 0.6 % / 0.07 % |
The all-distinct third cumulant needs rank about 3n/4 and the full depth.
3. Why it ends near 3.8e-9: cost
Each (source, layer) pair costs 4 logical n^3 products (2 transports + 2 readouts), about 120 pairs per net. The graded engine bills 190 u (q 0.186 on a steady predict), 80 % of it in Strassen-Winograd products, 15-17 u per layer at layers 7-14. The leaders’ public q histories imply about 14 dense n^3 per layer for everything; this representation needs about 30. Every physics lever we measured costs more q than it returns at this price (e.g. AGE_OLD 8: x0.925 MSE for +0.033 q; edge history: x0.97 for +0.02-0.05 q).
4. What reached the board
v30b 4.975e-9 → v33s7 4.4928e-9 → v35c1 4.4112e-9 → v35m2 4.2443e-9 → a_k31m 4.0775e-9 → c3_ab 3.9191e-9 → c9n 3.8792e-9 → c9z2f 3.8132e-9. Best graded MSE: 1.9951e-8 (edge history, q 0.267).
5. Grader pitfalls, with what happened
- Runtime margin under load: the same file graded clean (#336225) and then lost about 10 % of the suite to zero-prediction fallbacks (#336349) a day later. At 5 instead of 6 Strassen levels (local wall 60 s instead of 96 s, residual 0.66 s instead of 0.72 s, q +4 %) it graded clean (#336360). One failed MLP in 100 costs about 0.009 of score.
- The sandbox imports only an allowlisted stdlib (an
import bisectfailed two uploads); setup-time arena allocation and a full-level first predict zeroed uploads; uploads 1.15-1.35x slower than our base sat 80-90 min and returned “Evaluation could not complete”. - The (3,1) slice orientation: an oracle
U[i,j] = k4(z_i,z_i,z_i,z_j)must enter asU^Tin a table indexedk4(a,b,b,b); the wrong orientation turns a x0.909 lever into x1.020.
6. Method notes that saved us time
- Inject the exact quantity before building a cheaper estimate of it.
- Score combinations and rank a term by its effect on the scored output, not by the error of its channel: a better g4 channel left the final MSE flat (x1.002 on 6 nets).
- Check oracle levers against independent truth: with the official 1e9-path means the exact d3 + g4 lever was x0.520
(same-sample scoring x0.564), so it is not noise tracking.
Attribution and LLM use
Base: 504aldo’s V29 (MIT); the shipped file
WRITEUP_PHASE2.pdf (751.0 KB)
s of later uploads carry its notice. The author (Nurlan Abdrash, team NullPointer) set the research directions, chose and approved every experiment and upload, and checked the numbers. LLM coding agents (Anthropic Claude Opus 5.5 and Claude Fable 5.1) wrote the code, ran the experiments and drafted the text; external chat models served as theorists, and every claim was tested numerically before use. Details and the source of every number: attached PDF, section 8.