Our Phase-1 score is not competitive. We are posting because of what we did instead of tuning it, and because the negative results are measured rather than argued.
The contribution is a protocol. Take your estimator, replace one internal quantity with its ground-truth value, leave everything else running, and measure. The result is an upper bound on every idea whose effect passes through that quantity — including ideas nobody has written yet. If the bound is below your target, the family is closed and no cleverness inside it will help.
Each such measurement cost about fifteen minutes on one GPU. Together they closed 54 method families, most of which we would otherwise have implemented. The ones that matter:
substitute the exact … ceiling what it closes
per-step formula error ×1.19 γ-schemes, quadrature, symbolic
search, rank-r kernels, learned steps
mean and covariance ×3.36 the entire two-moment class
covariance alone ×1.10 every "better covariance" idea
per-network personalisation ×1.03 all per-instance constants
exactness through 3rd degree ×3.4 what third-order exactness buys
exactness through ANY degree ×7–9 EVERY smooth approximation, any price
The second row is the one that matters for this competition: a perfect two-moment closure — exact m and C at every layer — scores about 5th place, ×7.7 short of the 2nd-placed entry. The class most of us chose does not contain a winning method, and that can now be stated with a number.
Six results from the same corpus that may be useful to others here, each with its controls in the write-up:
1. A tuned closure is a balanced system of cancelling errors. A model predicts the per-step error of σ at R² = 0.836 — and correcting it yields exactly nothing (optimal coefficient 0.00, unchanged across a full grid). Substituting the true covariance diagonal makes the answer 8.5× worse. Any evaluation of a “more accurate submodel” must re-fit the system’s remaining constants jointly, or it measures their staleness instead.
2. The residual is the ReLU kinks — and the kinks are the medium, not a feature. Replacing ReLU by softplus_β collapses the Hermite tail (degrees ≥ 7) from 0.190 to 0.021–0.059 at β = 4, while the function is still 55–66% nonlinear: a kinked function has a power-law Hermite tail, so every smooth approximation family buys the same first three degrees and stops — ×3.4 for third-order exactness, and ×7–9 even for exactness at every order. The natural response — work with the singularities: cells, Gegenbauer, poles, sampling aimed at the switching hyperplanes — is equally dead, and for the opposite reason. The median distance from a sample to the nearest switching surface is 1.1e-04, and 99.1% of the probability mass lies within 10⁻³ of one. There is nothing to localise. (Forced by counting ~8000 hyperplanes; the layer-0 control reproduces |N(0,1)| quantiles to four digits.) Separately, six theories of approximation reduce to one number: in 256 dimensions only ridge functions integrate exactly, and the ceiling of any additive one-dimensional class is ×1.89 — equal to what polynomials already give. The bottleneck is additivity, not the basis.
3. The answer is born early; the error is made late. An exact identity (E[ReLU(p)] = E[p]·ḡ + c, c = Cov(p, gate), seal 5.35e-09) puts 50.1% of the answer in layers 0–7. The backward operator that carries it has effective rank 2.74 at layer 0 and condition number 10³¹ — three of a shallow layer’s 256 numbers reach the output. The quantity you want is assembled early and then transported through thirty steps that destroy all but three of its dimensions. This closes state reconstruction, shallow-layer refinement and targeted per-neuron correction at once, as a property of the network rather than of any method.
4. Two statements about the scoring rule rather than about any estimator. Under this rule the score of a pure Monte-Carlo estimator does not depend on the budget at all — score(u) = u·V·E²/(V+E²) with V ∝ 1/(uB−P), so as the fixed cost goes to zero the score tends to a constant. Seal: u = 1, N = 65000 and u = 0.1, N = 6500 return 6.741e-07 both times. Two consequences we did not expect: a better deterministic branch makes the optimal budget floor stricter, and the analytic advantage is a small-sample phenomenon — re-fitting every constant honestly, our advantage over plain sampling falls 4.24× → 1.47× → 1.09× at ×1 / ×8 / ×64 samples, and at ×64 the optimal weight on the analytics is exactly zero.
5. Two impossibility results, for anyone whose first instinct is to train something. The learned-surrogate family is closed at six independent corners, and the centre of it is counter-intuitive: adding capacity makes the estimate 200× worse (a wide linear least-squares student has bias 1e-07, the same student with nonlinearity has 2e-05), because least squares gives an unbiased mean for free and SGD does not. An offline-trained hypernetwork — which escapes the cost argument entirely — fails with a diagnosis: the cheap analytic base has essentially the same value for every network (rms ≈ 0.55 while the truth varies 0.50–1.36), because mean-field variance collapses to a fixed point under criticality. No cheap descriptor of an instance exists from which its answer can be recovered. Separately, per-instance personalisation is closed by unobservability: the optimal per-network constant depends on a linear functional of the realised noise, while every statistic computable inside the estimator is quadratic in it — the correlation is +0.714 across networks and −0.830 within one.
6. The complexity has a source, and it is two design decisions. The benchmark sits exactly on a critical point: weights are sign-symmetric (50.003% negative) and there are no biases, so ReLU switches off 50.2% of neurons at every layer — the maximum-activity point of the filter, where no scale separation exists. This is not a description, it is a lever. Take the same weights and flip a quarter of the negative ones to positive (renormalising the layer gain, or you measure scale rather than criticality): the effective rank of the ensemble of realised linear maps collapses 200 → 9.6, and flipping all of them gives exactly 1.01 — the linear network, solvable in closed form to a relative 1.7e-6. That rank in turn predicts how badly the Gaussian closure does (corr = +0.80 across 50 networks). Two decisions — sign symmetry and b = 0 — generate the entire difficulty, and b = 0 is a cusp rather than a slope: a bias of 0.15 makes the problem six times easier in one step, and extrapolating back from b ∈ [0.15, 1.5] underestimates the error at b = 0 by ×7.1. This is why experience with real trained networks does not transfer here: LayerNorm and biases de-tune them away from criticality.
We also report what we got wrong. Our corpus holds 396 recorded findings; 113 carry a correction to an earlier conclusion of ours and 51 are outright refutations of our own results. The write-up sorts them into the six mechanisms that generated them — including a stand that printed its own noise floor at a layer where the answer is exact by construction, and which three downstream stands then ignored while operating two orders of magnitude below it.
Attached, two documents:
-
Volume I — the write-up (
WRITEUP_PHASE1_EN.pdf, 38 pp): the protocol, the ceilings it produced, the mechanisms behind them, what worked, and what we retracted. -
Volume II — the compendium (
COMPENDIUM.pdf, 27 pp): the registry, in the form claim → number → seal → verdict, for all 54 closed classes and all 396 findings, plus the instruments and the seals that caught an instrument rather than confirming it.
Both derive from the same corpus: 109 working sessions, 600 measurement stands. Everything is measured either on the official grader — with a blend knob set to isolate one branch — or against the public dataset, whose final_means are exact at N = 10⁹. Where a measurement contradicted an earlier claim of ours, both appear. The report was drafted in Ukrainian and the fuller original (106 pp, with the derivations in long form) is available on request.
Happy to answer questions about any individual number, or to hand over the stand that produced it.
COMPENDIUM.pdf (762.4 KB)
WRITEUP_PHASE1_EN.pdf (1.0 MB)