There is a bug in flopscope.numpy.linalg.solve that undercounts batched right-hand sides

inside <|Ω|> is LLM generated bug explanation and minimal reproduction.

<|Ω|> When the coefficient matrix is 2-D and the RHS has a leading batch dimension, solve broadcasts the coefficient across the batch and performs one independent solve per batch — but flopscope charges only for a single batch. The cost appears to be derived from the coefficient’s shape and ignores the RHS batch dimension.

Minimal repro (flopscope 0.8.0rc5):
——————————————————————————————————————————————————————————————————————————————————————————
import flopscope as flops, flopscope.numpy as fnp, numpy as np
C = fnp.asarray(np.random.randn(256, 256))
B1 = fnp.asarray(np.random.randn(256, 1024))
B40 = fnp.asarray(np.random.randn(40, 256, 1024))

with flops.BudgetContext(10**15) as b: fnp.linalg.solve(C, B1); print(b.flops_used) # 145,402,538
with flops.BudgetContext(10**15) as b: fnp.linalg.solve(C, B40); print(b.flops_used) # 145,402,538
——————————————————————————————————————————————————————————————————————————————————————————

Both are billed identically, yet B40 does 40× the arithmetic — the 40 batch results are distinct and correct, and wall-clock scales ~40–50× (≈15 ms vs ≈760 ms here). So a batched solve can be charged up to the batch factor too little. This lets a submission run a large deterministic-sample estimator (rewriting each layer A @ W as solve(inv(W.T), A.T) over a batched RHS) while being billed for a small fraction of the real FLOPs. linalg.inv-based batched paths likely share the issue. Flagging as an exploitable measurement gap under the accounting rules. <|Ω|>

2 Likes

Also, the current top submission exploits this hack. Is there a way to retract it or remove it? I couldn’t find one.

2 Likes

Hi @williawa ,

Thanks for the report.
We have replicated this issue, and will be folding in a fix for this into the upcoming release.

Retraction of submissions is not possible, however, once the fix is in, the affected submissions will automatically be evaluated using the patched flopscope.

Thanks,
Mohanty

2 Likes