WhestBench_Phase2_writeup_Davis_submission_337276.pdf (1.2 MB)
Hello,
Please find attached my Phase 2 technical write-up for the Algorithmic Contribution Prize.
- Submission ID: #337276 (Q71q-a), graded on the Phase 2 evaluator, public score 3.062e-9
- Write-up: WhestBench_Phase2_writeup_Davis_submission_337276.pdf (34 pages)
I apologise that this reaches you after the 28 October deadline; I had the write-up finished but failed to send it. I understand if it can’t be considered, but I’d be grateful if it could be.
The write-up includes a full disclosure of LLM use (Anthropic’s Claude and OpenAI’s Codex), as the guidelines ask.
Thank you,
Matthew Ryan Davis
Post-contest follow-up (not part of the #337276 submission)
After Phase 2 closed I took the same cumulant machinery to a harder question: estimating the probability that a trained network fails under rare input noise. The short version is a cautionary result, attached as a 14-page PDF: “Steered rare-event estimators can miss failure regions: evidence from trained ReLU networks.”
- Setup. Contest-shape MLPs (1024 wide, 16 layers) trained on MNIST, including label-noise variants. The task is estimating rare misclassification probabilities under Gaussian input noise.
- Main finding. White-box shortcuts (FORM/SORM, Gaussian closure, importance sampling steered by the network’s internals) can look confident while missing whole failure regions, and their own diagnostics (relative SE, effective sample size) can fail to flag it. Subset simulation, a splitting method, was the more reliable comparator.
- Also included. Smaller exploratory applications (backdoor screening, a training-health monitor) that came out negative, reported as such.
Same disclosure as the main write-up: the experiments and drafting were done largely with LLM agents (Anthropic’s Claude and OpenAI’s Codex) under my direction; claims are hedged where we did not validate them. Feedback very welcome.
Davis_steered_rare_event_estimators_post_contest_paper.pdf (345.1 KB)