Memory usage limit

ARC White-Box Estimation Challenge 2026
#1

When submitting an estimator that creates intermediate arrays larger than ~100 MB, the grader raises:

ValueError: result array too large: 268435456 bytes exceeds 104857600 byte limit

This originates from flopscope/_remote_array.py on the evaluation server — a module that doesn’t exist in the local flopscope installation, so the limit is invisible during local development.

After checking the starter kit docs, whestbench source, and the challenge page, I couldn’t find this constraint documented anywhere. The only documented resource limits I found are:

  • FLOP budget: 272B FLOPs
  • Submission tarball: 50 MiB
  • Run timeout: 60s

Questions:

  1. Is the 100 MB per-result-array cap intentional and fixed?
  2. Could it be added to the docs, so participants can design their chunking strategy accordingly?
1 Like
#2

Hi @RomanChernenko,

Thanks for reporting this — we’ve shipped a fix.

That 100 MiB limit was a per-array memory guard in the grading sandbox (applied to both your inputs and your predict() output, during smoke and grading), there mostly for operational reasons. We’ve raised the per-array cap to 4 GiB (and the live-array-count cap to 10 million) — generous enough that an efficient estimator won’t come close, so memory shouldn’t be a constraint in practice. If these caps still feel restrictive for your approach, drop us a line at arc-whestbench@aicrowd.com with the details, and we would be happy to reconsider these caps.

We re-graded your two affected submissions: 311832 now grades fine under the new cap. 311812 still hits the same error even at 4 GiB, so it’s worth chunking your inputs and outputs into smaller arrays - processing in row/column blocks keeps memory bounded and is essentially free against your FLOP budget, since reshapes and allocations cost 0 FLOP.

Other submissions in Phase 1, from other teams, that hit this error have also been re-graded under the new cap.

Best,
Mohanty

1 Like
#3

Thank you for the fix. The submission id 311812 failed because of using non-supported NumPy operation, so everything else looks fine.

1 Like