Submission ID: #337389: 3.82e-9 adjusted, 1.94e-8 final-layer MSE, 19.75% utilization. Team whest_winner.
- Write-up: my-phase2-writeup.pdf (316.7 KB)
- Code: ARC-WHEST-Challenge (MIT). It is the starter kit plus one commit holding the submitted code, identical apart from comments and docstrings.
What this is, and what is mine
The estimator is a sample-free factorized K=3 cumulant chain with a small learned corrector on top. Most of the method is other people’s, and I credit it first:
- Chain method: Wu et al. 2026 (paper, ARC code).
- Phase 2 chain design: team 504aldo (code).
- Corrector design: @Susupro’s LT2 (topic 18254, code).
All three are MIT, and their notices ship in every file. My contributions, each with a graded or paired measurement:
- Cost engineering of my independent port: 6.4e-9 → 4.48e-9.
- Utilization fell from 31% to 20.6%, with raw MSE roughly flat.
- The steps were Strassen–Winograd routing, grouped QR joins, symmetric weighted Grams, rank schedules, an adaptive λ and pairwise κ₄ coupling.
- Each step was graded on its own.
- Diagnosis and repair of a residual-time failure that was blocking a −7% hub-pruning gain. The repaired chain graded at 4.25e-9.
- The LT2 corrector, ported to my chain and evaluated frozen on 200 held-out networks.
- That panel predicted the graded change exactly: 0.899× predicted, 0.899× graded.
- My readout-weighted variant was worse, which is a negative result.
- An oracle map of where the chain’s error lives.
The chain without the corrector was graded too (4.25e-9). The corrector is confined to lt.py, lt_weights.py and a few hooks in estimator.py, so its impact can be read straight off the two scores.
Findings
1. The residual cap is per client operation, not per FLOP.
Four of my cheaper chains failed on the grader’s wall-time or residual caps. The last added dead-row hub pruning (−7.07% adjusted locally) and failed on 21 of 100 networks.
Through the public flopscope 0.12.3 client and server:
- The pruned chain was not slower than the one that had passed: 0.440 s against 0.438 s per call. Both sat near the 0.4 s cap.
- One prediction makes about 65k client round trips. Each costs about 4 µs of residual, rising to about 24 µs after a long server operation.
- Recursive Strassen adds a fixed set of operations per level, however small the product.
The fix: cap the Strassen level by each product’s dense work (level 5 at ≥ 4e9 FLOPs, level 4 at ≥ 1e9, else 3) and cache pool slices.
| Chain | Client residual per warm call | Graded |
|---|---|---|
| engineered chain | 0.438 s | 4.48e-9 |
| + hub pruning | 0.440 s | failed (residual) |
| + fewer client ops | 0.357 s | 4.25e-9 |
2. The corrector transfers at full strength, and a frozen held-out panel predicts its grade.
I trained the 54k-parameter corrector on 3,048 trajectories of my own chain (the public full split plus Keenan Pepper’s corpus), selected the epoch on mini, and committed the weights. Only then did I read the truth for 200 unused corpus networks.
| Raw ratio [95%] | Wins | FLOPs | Adjusted | |
|---|---|---|---|---|
| Frozen panel | 0.8877 [0.8818, 0.8936] | 199/200 | +1.24% | 0.899 (predicted) |
| Grader | 0.890 | +1.3% | 0.899 |
@Susupro’s chain gets 0.900× from the same design, so the port is complete.
3. Readout-weighting the corrector’s loss does not help. Weighting each layer’s local loss by how strongly its error reaches the output made the result worse: +0.21% [+0.07, +0.34] on the frozen panel, better on only 89 of 200 networks.
4. Float16 features with float32 deployment produce NaN.
- Columns that are constant in training (for example Φ(0) = 0.5 at layer 0) get the 1e-6 scale floor.
- Float32 inference differs from the float16 mean by 2e-3, so z ≈ 2000, and the state overflows.
- Fix: mask the scale (
where(sz > 1e-5, 1/sz, 0)) rather than flooring it.
5. Given exact marginals, the chain is nearly exact.
I injected 1e9-sample reference values at every layer, on 20 development networks:
| Oracle | MSE change |
|---|---|
| variance | −39.5% |
| κ₃ | −24.3% |
| κ₄ | −19.9% |
| all three | −97.0% |
Yet every hand-written κ₄ repair made the score worse (+0.1% to +67%). The adaptive λ, the κ₄ coupling and the κ₂₂ model are calibrated to the chain’s κ₄ deficit. My guess: a learned corrector that sees the whole state can fix what slot repairs cannot.
Why 3.82e-9 and not 3.1e-9
Compared against @Susupro’s chain, mine is 1.216× theirs (4.25e-9 against 3.494e-9). The corrector factor is the same for both, so the whole gap is in the chain:
- About three quarters is compute: 19.5% against 16.6% utilization.
- About one quarter is raw MSE: 2.18e-8 against 2.09e-8.
Mine lacks their SSC closure, young-atom pruning and level-6 hub.
Grader pitfalls
- Measure residual through the public client and server, with warm calls and a matched baseline. In-process timings say nothing about the cap.
- Repeated-operand
einsumvalidates symmetry, and float32 roundoff in A·diag(d)·Aᵀ can fail that check remotely. Scaling both factors by an exact power of two avoids it. - Compare
fnp.dtypeobjects, not strings. A failed guard silently falls back. - A local pass is not a remote pass. One of my chains passed at the same local residual level at which its successor failed a month later.
Attribution and LLM use
LLM coding agents (mainly Anthropic’s Claude in Claude Code) wrote nearly all of the code, ran the experiments and drafted this post and the PDF. I set directions, approved every experiment, download, cloud spend and upload, made the submissions and checked the numbers.
Thanks to the ARC authors, @504aldo , @Susupro , koushik_rudra (V34) and @keenanpepper (corpus).