šŸ’¬ Feedback & Suggestions

ARC White-Box Estimation Challenge 2026
#1

We are constantly trying to improve this challenge for you and would appreciate any feedback you might have! :raised_hands:

Please reply to this thread with your suggestions and feedback on making the challenge better for you!

  • What have been your major pain points so far?
  • What would you like to see improved?

All The Best!

#2
#3
#4

Is the smoke test flop budget lower than the full submission budget? Iā€™m seeing submissions run out of budget in the smoke test even though they are under the cap locally, so I suspect it may be half what it should be

#5

Good catch, @jamespayor - confirmed and fixed. The smoke test was running under a stale hardcoded budget (1e10) instead of the real grading budget (6.8e10), so estimators spending anywhere between the two were getting wrongly rejected. The fix is live now (smoke uses the exact same per-MLP budget as grading), and we have re-graded the submissions that were affected, so they should show up scored. Thanks for flagging it!

Affected submissions that were re-evaluated: 309660, 309713, 309714, 309723, 309724, 309738, 309742, 309744, 309745.

#6

Nice, thank you for the fix and rerunning! :slight_smile: