This should be pretty easy to figure out, but the six current leaderboard submissions that are tied at 3.93e-7 adj / 3.93e-6 MSE are probably all the plain & simple covariance propagation baseline. Which decisively beats sampling in Phase 2. Exciting!
Which means the 8 leaderboard entries doing even worse than that are kind of hilarious.
The leaderboard is already telling a pretty entertaining story. If the tied 3.93e-7 / 3.93e-6 entries really are just the vanilla covariance propagation baseline, then Phase 2 is a great reminder that “simple” can be surprisingly hard to beat.
And those eight submissions below the baseline… perhaps the covariance propagation method deserves a little more credit than we initially gave it.
The fact that a straightforward covariance propagation baseline is competitive across multiple leaderboard slots tells you the challenge is well-designed. Participants can’t just throw complexity at it and win—they need to genuinely outthink the math.
As for the bottom 8: great motivation to debug before submitting 