@omw : The evaluators, as we launch Phase 1, currently use: flopscope v0.8.0rc1 and whestbench v0.12.0rc0.
This might change, as we address the feedback we receive over the next week - (more details here)
For every submission, you can also check out the exact versions that the evaluator uses, in the Configuration tab of the Submission Details page.
(I also noticed that plain numpy isn’t importable in the grader env—is that by design?)
Yes, that is a challenge design choice for whestbench.
We might revisit that decision if we come across strong usecases where using numpy (which counts towards the heavily penalized residual time) is necessary. So we are open to feedback.
Best,