Phase 1 Write-Ups, Team Puffi

Hi guys,

posting the algorithmic submission for Team Puffi:

Note, this is a mechanistic approach, and we base it on Wu’s paper.

The ARC’s cumulant propagation algorithm is an example of deduction-projection estimator. We play a little bit with the projection step (and we add some fitting, but we do not think it is the end of the world, as in several areas of physics sometimes you absorb complex processes in some coefficients).

We do not do great with this submission as we get an adjusted score of \sim 1.65\times 10^{−6} :slight_smile:

But we really wanted to put some effort in trying something from a more mechanistic side.

On the other hand, we also post a score short write-up submission, just in case we end up among the top (you never know)

This one is much better, with a top adjusted score 9.10 \times 10^{-8}. However, it is based on a mix of analytical and QMC methods, and there is a lot of engineering and optimizations (we did our best to not game flopscope!!).

We are really excited about this second approach: if we find a way to select Kerdock frames, or even invent new ones, based on mechanistic understanding of the network, we think we can do better at a much lower cost (0.1-0.2 of mult) (some sort of quadrature rule based on your specific function). We still did not solve this, but we hope to tackle this too!

AI DISCLAIMER: our team used AI during the preparation of the manuscripts, as well for coding, research, and brain storming.

Sincerely speaking, I am very curious about how you all are keeping up with AI output. I ended up several times with research debt :frowning: And a few times it was difficult to come back on top to have a big picture. Would be curious about your harness! For now, I only have an experiments folder, keep a diary of main plots, and use some simple notes. I also found the new models, like Opus 5, Fable, and GPT-5.6, are quite…cryptic and sometimes try to avoid hard work. Not sure if this is due to RL, or something else, but this was an additional frustration of our journey.

5 Likes

Subagents saved me alot, I specifically told them to delegate to the lesser models, or even better the local ai (I have a RTX8000 which helped but it’s a bit dated compared to probably some monsters i imagine some have) when possible and doesn’t diminish quality. I still ran out of tokens ALOT though despite this. ESPECIALLY when I gave codex the /goal command and check hours later and am like shit, I forgot to tell him to take it easy on the tokens.

Oh, and azure, (and i think a fw other services like aws and whatnot but i really liked azure), offers a free first month $200 it was awesome, I’m going to miss that but that saved me from having to use ai for some of the research as weell: https://azure.microsoft.com/en-us/pricing/purchase-options/azure-account

1 Like

Thanks for this, I did not know about it! I used runpod a few times as my laptop died due to OOM. $200 sounds quite a lot!