Threads 18099 and 18108 have already described, publicly and with measurements, that arrays reachable through flopscope.numpy can be operated on without those operations being registered as instrumented FLOPs, so the work is charged only through the residual wall-clock term at 1e11 FLOP/s. I am not adding anything new here or asking about any particular submission but I was wondering where the boundary is, because it seems like two parts of the rules point in different directions.
Section 5.2 and section 5.6 state that bundling native code is permitted, and that libraries which are not pre-installed may be shipped and are charged through wall time. Read on its own, that suggests doing real arithmetic outside the instrumented path is an intended and priced tradeoff.
Section 12 lists “attempts to evade flopscope accounting” as grounds for disqualification. The challenge announcement gives as its example of unambiguous hacking “modifying constants or counts held in memory.” The behaviour above does not modify any count. It never reaches the counter at all.
So my question is:
- Is arithmetic performed on an array obtained from flopscope.numpy, where that arithmetic does not pass through the instrumented path and is charged only via residual wall time, inside or outside the intended accounting boundary?
- Does the answer change if the array is obtained deliberately rather than incidentally?
- Does the answer change between Phase 1 and Phase 2, given that flopscope 0.10.0 shipped on 30 July and section 5.3 permits re-grading under a corrected grader?
Thank you!