Algorithmic contribution prize guidelines: How ARC judges these prizes — discretion, technical writeups & LLM usage

ARC White-Box Estimation Challenge 2026
#1

Guidance from the Alignment Research Center (ARC)

In addition to score-based prizes, there are prizes for the best algorithmic contribution:

  • Phase 1: $10,000
  • Phase 2: $20,000

These prizes will be awarded at ARC’s discretion to the method we think most improves our understanding of white-box estimation for random MLPs. We are most interested in “mechanistic” estimation methods, as discussed in our blog posts on competing with sampling and mechanistic estimation for wide random MLPs. We are less interested in methods that rely heavily on sampling, fine-tuned constants, careful performance optimization, and opaque LLM-optimized code (although clever sampling-based methods are of interest if they rely on interesting structural observations, and LLM-written code is of interest providing it can be deciphered).

Technical writeups. The chance of a submission receiving an algorithmic contribution prize is greatly increased by the inclusion of a technical writeup explaining the algorithmic approach used and how it was developed. We will likely start by reading the technical writeup for the highest-scoring submissions, and award the prize to the submission where novel “mechanistic” ideas made the largest improvement to performance over previously-known methods.

LLM usage. We are ultimately interested in the quality of the algorithmic contribution itself, regardless of how it was obtained, and LLM usage is encouraged. However, contestants should be fully transparent about the extent to which LLMs were used to generate code and/or portions of technical writeups. If contestants have significant uncertainty about how and why their code actually works, the relevant portions of the technical writeup should be appropriately hedged and/or labeled as guesswork (for example, “the LLM gave this explanation, which we did not validate/which we validated by …”). If we notice unhedged, dubious claims, then we are likely to be more skeptical about the remaining content and may skip over submissions entirely.

Update 6th July, 2026: Details on how/what to submit, included here.

1 Like
#2
#3

Hi, and thanks for the detailed clarification on how the algorithmic contribution prize is judged.

I have a graded Phase 1 submission on the leaderboard (#314331) and have prepared an accompanying technical write-up documenting the approach, its validation, and the negative results along the way.

Could you clarify the mechanics of submitting the write-up for consideration? Specifically:

  1. What is the intended channel — an AIcrowd attachment/upload field on the submission, a post in this forum, a PDF link, or another process?
  2. Is there a deadline for the write-up (e.g. must it accompany the submission by the Phase 1 close), or a separate window?
  3. Should the write-up be tied to one specific submission ID, or does it stand on its own?

Thanks — I imagine this would be useful for other participants preparing write-ups as well.

#4

Hi @pscamillo - great questions, and thanks for preparing a write-up. Here’s how Algorithmic Contribution Prize submissions will work, for both phases:

What to submit: A PDF technical write-up + exactly one submission ID. Both are mandatory, and the mapping is strict: each write-up must correspond to a single submission that was successfully evaluated by the grader of that phase (i.e., a graded, non-failed submission on the Phase 1 evaluator for Phase 1 prizes, and likewise for Phase 2). Since the submission ID points to a validated code tarball already on our servers, nothing else needs to be uploaded - the write-up should describe the approach behind that specific submission and the evidence/results supporting it (negative results and ablations welcome).

How to submit (either channel works):

  1. Privately, by email to arc-whestbench@aicrowd.com, or

  2. Publicly, as a post on the Challenge Discussion Forum - public write-ups will additionally be considered for Community Contribution Prizes, where applicable.

In both cases, clearly state the submission ID the write-up refers to.

Deadlines: You get one extra week after each phase closes to finish the write-up:

  • Phase 1: August 7, 2026, 23:59 UTC

  • Phase 2: September 26, 2026, 23:59 UTC

:warning: Note: the extra week is for writing only. Once a phase ends, its evaluator closes - you will not be able to make new submissions or re-grade anything for that phase. So the submission you intend to reference must already be successfully graded before the phase deadline (July 31 for Phase 1).

The judging criteria remain as described in the original post above (ARC’s discretion, mechanistic ideas prioritized, transparency about LLM usage). We’re updating the challenge page to reflect these mechanics.

#5

Thanks again for the clarification. I’ve posted my Phase 1 write-up for submission #314331 here: [Phase 1 write-up] Characterizing a systematic scale bias in the Gaussian-closure estimator (submission #314331)