Hey everyone! I am sharing my Phase 1 write-up for submission #327749.
I joined the challenge late, but the campaign ended up taking a useful turn. I initially thought the 128-basis Kerdock carrier worked because it was a tight spherical 5-design. Directly auditing the construction showed that it is not: one basis is missing. Correcting that mistake led to a stronger and more specific explanation.
The paper’s main contribution is an exact ReLU-specific theorem. For antipodal quadrature under the infinite-width He-ReLU kernel, pairwise mutually unbiased orthonormal bases minimize expected quadrature MSE among all equal-sized unions of orthonormal bases, at every positive depth.
We test the theorem’s predictions on 200 validation networks and an untouched 200-network lockbox, each across 16 fixed global realizations. The predicted ordering transfers cleanly to finite width, and the missing-basis prediction is remarkably close to the protected result: 1.00930× predicted versus 1.00907× measured. In the matched equal-path study, the MUB carrier improves corrected MSE by 1.286-1.391× over the Haar, randomized-flat, and Owen-Sobol controls.
The second contribution is computational. A 1,024-path gate-support pilot allowed the submitted estimator to remove about 8.3% of effective compute with only 0.07% raw-MSE change. A controlled threshold sweep also reveals a sharp support-removal cliff: rarely firing units can carry disproportionate downstream value.
The write-up connects the exact theorem, protected finite-width experiments, and the successfully graded estimator. It also includes the negative experiments and limitations that helped separate the actual mechanism from several plausible but incorrect explanations.
I believe the combination is a meaningful algorithmic contribution: a new optimality result tailored to deep ReLU expectations, together with a practical method for turning that structured quadrature into a lower-cost estimator.
Huge thanks to the organizers and everyone who shared ideas, results, and corrections. This was intense, fun, and taught me a lot. The PDF is attached below. Feedback, corrections, and criticism are very welcome.
phase1_algorithmic_contribution_327749_final.pdf (399.7 KB)