Cost-model question: releasing temporaries of counted ops is charged to residual — and a request to expose out= on fnp.matmul

ARC White-Box Estimation Challenge 2026
#1

When a counted op returns a large array and the caller later drops it, the native buffer release happens between wrapped calls, so it lands in residual_wall_time_s and is billed at λ = 1e11 FLOP/s in effective compute. The op’s own NumPy call is correctly attributed to flopscope_backend_time_s, and the docstring at _budget.py:666-674 does say user Python, GC and uninstrumented NumPy count as residual — so we are not claiming a timing bug. The release of a participant-requested result is a real cost that has to be classified somewhere. Our question is whether charging it as participant residual at the λ rate is the intended policy, given that it is allocator- and host-dependent, and that the common result = op(...); consume; drop pattern currently cannot avoid it, because fnp.matmul exposes no out=.

Minimal reproduction (flopscope 0.9.1, numpy 2.2.6):

import flopscope as flops, flopscope.numpy as fnp

def run(keep, n=300):
    ctx = flops.BudgetContext(flop_budget=int(1e14), wall_time_limit_s=3600.0, quiet=True)
    with ctx:
        a = fnp.zeros((2700, 1000), dtype=fnp.float32)   # 10.8 MB operands
        b = fnp.zeros((1000, 1000), dtype=fnp.float32)
        arena = fnp.zeros((2700, 1000), dtype=fnp.float32)
        kept = []
        for _ in range(n):
            t = fnp.matmul(a, b)          # counted op returns a 10.8 MB temporary
            fnp.copyto(arena, t)          # counted, billed 1.0*numel into F under 0.9.x
            if keep: kept.append(t)       # else: t released on rebind, inside the context
    print(ctx.residual_wall_time_s / n)

run(keep=False)   # ~100-120 us/op residual
run(keep=True)    # ~6-8 us/op residual   (release deferred outside the context)

On our machine (macOS/arm64) the free arm shows ~100–120 µs/op of residual vs ~6–8 µs/op for the keep arm — a 13–16× contrast. Controls: copying the same 10.8 MB between persistent buffers costs ~2 µs/op residual (the movement itself is billed to backend), and disabling cyclic GC changes nothing. Deferring release does not avoid the cost: clearing the kept list later, still inside the context, costs about the same (~150 µs/array). Since the mechanism is the allocator itself, the magnitude on the Linux evaluator hosts may differ from what we measure on macOS.

Why we think this deserves an answer:

  1. The charge attaches to a pattern participants cannot express any other way: fnp.matmul takes only (a, b), so “compute, place, release” is forced whenever a contraction result must land in an existing buffer. Notably, the support already exists internally: _einsum_routed_binary (_pointwise.py:2847-2906), which public matmul routes through on every call, implements out= completely — it includes the out-array’s dtype in billing, unwraps it, forwards it to the native op, and returns it as the result. None of its call sites currently passes out=, and the public matmul signature at :2957-2974 is (a, b) only. Exposing a correctly-billed matmul(..., out=) therefore looks like a ~two-line change, with billing semantics that already exist and would stay identical (contraction cost is computed from shapes either way). einsum already exposes out= publicly and bills it identically, but it is not a practical substitute: NumPy does not dispatch these batched contractions to BLAS through einsum, so it is far slower in wall time for the same billed cost.
  2. The cost is allocator- and host-sensitive, so identical submissions can price differently across evaluator hosts, and participants cannot measure this term reliably on their own machines.

Given that the billing semantics for out= already exist internally, exposing fnp.matmul(..., out=) looks like the simplest resolution: it makes the pattern avoidable, improves API consistency with the fnp ufuncs and einsum, and reduces allocation traffic and peak RSS per predict().