[Phase 2 write-up] Where the per-source leg representation of factorized K=3 ends (adj 3.8e-9), and the measurements behind it (submission #336225)

Submission #336225: adjusted 3.8132e-9, final-layer MSE 2.0994e-8, q 0.1816. Full write-up (PDF) attached.

This is a month of work on top of 504aldo’s public V29 (MIT, forum topic 18218), the factorized K = 3 cumulant propagation of arXiv:2605.05179 with a memoryless fourth cumulant. V29 is a strong base: 5.7x cheaper than the organizers’ factorized reference at a better MSE. We extended it with an all-distinct kappa4 edge term (free through an exact D21 = M W^T identity), a kappa3-covariance edge history (graded MSE 1.9951e-8, #336215), two exact term fixes, a fitted (3,1) law and a cost engineering pass. The main result we want to share is measured: where this representation ends, and why, so that nobody has to spend a month finding it out again.

1. The error budget (oracle injections into the production chain)

Exact values replace one carried quantity at layers 8-15; the number is the final MSE ratio. “16M” is a net scored against its own 16M-path means, “pub0” is public net 0 against the organizers’ 1e9-path truth.

exact at layers 8-15 16M pub0
var 0.687 0.647
C_off 0.778 0.819
D21 0.954
d3 + g4 0.610 0.508
var + d3 + g4 0.190 (oracle floor) 0.127
(3,1) kappa4 slice, on the graded engine 0.909

The final error sits in three per-neuron inputs of the mean map (var, d3, g4) of near-gate neurons at layers 8-15; the whole covariance side made exact leaves 70 % of it. d3 and g4 errors are the all-distinct kappa3 / kappa4 memory. The channels cancel: an exact (2,2) slice alone raises the MSE (1.033), and the LS-fitted (3,1) law works best scaled by 0.6-0.8.

2. What the memory cannot lose (measured)

change to the kappa3 memory MSE
all-distinct part set to zero (n = 256 / 512 / 1024) 13x / 23x / 43x
Tucker rank 256 / 512 / 768 13.5x / 3.7x / 1.36x
drop sources older than 3 / 5 / 8 / 12 layers 43x / 19x / 5.0x / 1.25x
old sources merged into one exactly transported slice plane (K = 4 / 3 / 2) 7.2x / 11.7x / 17.8x
young A leg compressed to rank 256 (both slices) / 768 (D21 slice only) 14.5x / 1.035x
old legs in one carried basis, rank 256 / 384 / 512 (rel. error of their slices at layer 15) 2.4 % / 0.6 % / 0.07 %

The all-distinct third cumulant needs rank about 3n/4 and the full depth.

3. Why it ends near 3.8e-9: cost

Each (source, layer) pair costs 4 logical n^3 products (2 transports + 2 readouts), about 120 pairs per net. The graded engine bills 190 u (q 0.186 on a steady predict), 80 % of it in Strassen-Winograd products, 15-17 u per layer at layers 7-14. The leaders’ public q histories imply about 14 dense n^3 per layer for everything; this representation needs about 30. Every physics lever we measured costs more q than it returns at this price (e.g. AGE_OLD 8: x0.925 MSE for +0.033 q; edge history: x0.97 for +0.02-0.05 q).

4. What reached the board

v30b 4.975e-9 → v33s7 4.4928e-9 → v35c1 4.4112e-9 → v35m2 4.2443e-9 → a_k31m 4.0775e-9 → c3_ab 3.9191e-9 → c9n 3.8792e-9 → c9z2f 3.8132e-9. Best graded MSE: 1.9951e-8 (edge history, q 0.267).

5. Grader pitfalls, with what happened

  • Runtime margin under load: the same file graded clean (#336225) and then lost about 10 % of the suite to zero-prediction fallbacks (#336349) a day later. At 5 instead of 6 Strassen levels (local wall 60 s instead of 96 s, residual 0.66 s instead of 0.72 s, q +4 %) it graded clean (#336360). One failed MLP in 100 costs about 0.009 of score.
  • The sandbox imports only an allowlisted stdlib (an import bisect failed two uploads); setup-time arena allocation and a full-level first predict zeroed uploads; uploads 1.15-1.35x slower than our base sat 80-90 min and returned “Evaluation could not complete”.
  • The (3,1) slice orientation: an oracle U[i,j] = k4(z_i,z_i,z_i,z_j) must enter as U^T in a table indexed k4(a,b,b,b); the wrong orientation turns a x0.909 lever into x1.020.

6. Method notes that saved us time

  • Inject the exact quantity before building a cheaper estimate of it.
  • Score combinations and rank a term by its effect on the scored output, not by the error of its channel: a better g4 channel left the final MSE flat (x1.002 on 6 nets).
  • Check oracle levers against independent truth: with the official 1e9-path means the exact d3 + g4 lever was x0.520
    (same-sample scoring x0.564), so it is not noise tracking.

Attribution and LLM use

Base: 504aldo’s V29 (MIT); the shipped file

WRITEUP_PHASE2.pdf (751.0 KB)

s of later uploads carry its notice. The author (Nurlan Abdrash, team NullPointer) set the research directions, chose and approved every experiment and upload, and checked the numbers. LLM coding agents (Anthropic Claude Opus 5.5 and Claude Fable 5.1) wrote the code, ran the experiments and drafted the text; external chat models served as theorists, and every claim was tested numerically before use. Details and the source of every number: attached PDF, section 8.

1 Like