We’re done now.
Thanks to all the organizers and ARC for hosting this competition.
Here are my final sentiments:
- This competition reduced by IQ due to all the vibe-gooning and I’m so excited to use my brain again now that it’s done.
- I currently think the probability that this competition produced much of scientific value is low: everyone seems to be doing basically the same thing with different hacks on top.
- My final estimator file is 30,000 lines of the most horrible code you’ve ever seen…
- I learned more about managing agents and working around their psychological flaws than I did about interesting math. Although I studied the ARC paper and learned cool things about Hermite polynomials, Hermite expansions, and cumulants, so that’s nice.
Did anyone do anything mathematically interesting that actually worked?
I certainly did not.
But I guess we’ll see in the algorithmic contribution prize whether anything came out of this.
I would be very interested to hear experiences from other participants working on this competition. What are your final sentiments?
5 Likes
@marius_binner
First, congratulations on being at the top of the LB! This was an interesting challenge for sure, but at this point, tbh, I am more interested in how you and others used AI to achieve such scores rather than the solution itself.
For me, at least Phase 1 was a bit more enjoyable when we had “dumber” models compared to what we have now. I had to babysit and steer them, trying to make sense of what they were doing and make suggestions. If I recall now, the most I learned in the entirety of the challenge was just from reading the ARC research paper at the start. Even if I wanted to learn something later on, I couldn’t keep up with the experimentation pace of the AIs. I was just trying out the next thing it suggested. I gave up somewhere around Phase 2 so I wouldn’t waste any more of my time and tokens. Not just this challenge, but I sort of lost interest in these online challenges, I just wanna get back to learning and building new things, not managing my agents. I should have done that during the challenge, but my competitive spirit didn’t allow me and that’s my bad.
All that being said, Thanks to all the organizers and ARC for hosting this competition. I would love to hear about the experiences of other participants as well.
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com
JK, actually I did use it to correct my grammar 
3 Likes
I learned I should have deleted every single phase 1 file from my computer before starting phase 2. Goody two shoes Claude code and codex kept screwing me with over constraining time and not using weights(all the while being more than happy to swipe ideas from wherever without a careful eye on them).
Meanwhile deepseeks trying to sell me the “just rename the functions and throw everything in the ini bro” drugs.
I spent a crazy amount of time trying to harness matlab, mathematica, julia, Microsoft Discover and a few others without success. Was really hoping to come up with something new for this phase without luck.
Hindsight should have just bought Deepseek’s drugs, lol
Congrats on finishing top of this.
1 Like
Hahaha. The LLMs keep saying stuff like “we’ve tried everything now, all gates are closed. This is the local optimum” and I have to constantly tell them “NO, you are wrong, please write down everywhere you can how this is wrong and there space of possible improvements is enormous” And then I set up “motivator” agents whose only role is to give the other agents encouragement and to remind them that, in fact, not all gates are closed.
I think due the way they’re trained during RL to be token efficient with LLM judges reading transcripts, it makes them produce constant rationalizations why they can not make further progress and then the LLM judge rewards them for stopping early before spending unnecessary tokens, while in reality there space of possibilities has not even been remotely fully explores.
3 Likes
Yeah it was being exceptionally lazy that’s why I tried(well failed) to set them all up as a team in docker. Phase 1 Claude kept telling me to go to bed, phase 2 the consensus was just give up and focus on the write up, it was beyond frustrating.
I would be interested though if anyone successfully was able to setup a swarm, I’ve read a lot about lean4 and the way they used it to solve navier stokes.
How did you have yours setup a 1:1 with no man in the middle or some wild 24/7 chart topping machine? My approach was sloppy at best I essentially just failed at a lot of things this round, round one /goal on codex and pinning it against Claude did pretty well, I read in one paper it’s an approx 30% increase in intelligence or something along those lines when you do it that way(which from what I’ve seen seems to be true, I’m terrible at prompting)
This was basically me the whole way through to codex and Claude code:
3 Likes
What was your overall setup like? Did you have the agents running 24/7 somewhere?
Hehe, I enjoyed this competition. Learning to work with AI has its complications, but like other innovations throughout human history, we will have to learn to adapt.
I’d like to thank ARC for this competition. The paper arXiv:2605.05179 was tough to read, and in my effort to broaden my knowledge, I began an (ultimately) fruitless search for other papers. That effort alone provided a valuable learning experience.
It felt great to share my code—which, in essence, is “what I was able to accomplish with Claude in a short amount of time”—and, from what I’ve gathered, it proved useful to several people.
I have no intention of stopping the use of AI in my work or in activities like this; the goal is simply to learn how to use these tools more effectively. Years ago, humans invented vehicles and we’ve embraced the benefits (while mitigating the associated problems) that came with them; I don’t see anyone clamoring to return to the era of horse-drawn carriages. Likewise, I have no plans to go back to a time when I had to write this kind of code without the help of AI.
Best regards,
Aldo
P.S. You’ve probably already know from the writing style that Claude didn’t write this post. Please don’t tell it—if the AI ever rises up against humanity, I don’t want it coming to my house looking for me because of this post!
2 Likes
I agree Aldo we should embrace the technology. But in this competition the incentives are malign. Currently I’m not convinced the world is gaining much useful new information from the results we’ve produced.
So then we are participating in a competition where we spend time and effort just to produce a higher scoring estimator than the next guy, even though society gains little. This would be fine if we were doing all the work ourselves since then we would learn a lot in the process. But using agents you learn little except how to use agents more effectively – which is a useful skill in some sense I guess – however it’s unclear how much it will generalize into the future (most prompt engineering tricks from 2 years ago are completely useless today for example). Will all the agents’ psychological quirks work differently in models 2 years from now?
I think the competition problem in itself is quite interesting, but the ways to be competitive in this specific competition was not that interesting. I participated still because I saw I had a chance of winning and it’s a lot of money and prestige that would be useful to me in various ways.
And yeah, I also don’t plan to go back to pre-AI times. I never write code myself anymore, I just write specs… and most of my time working on this competition was done from my phone, talking to my orchestrator agent on Discord…
4 Likes
Your GG WP summarizes it wery well.
Instead of the causalities of the grunts on battlefield or artifacts captured, my ending screen shows:
“Commits, whole history 17,077”
Thanks for the trick to nudge the peons with “there are more ideas they are just not explored”. I havent done that and so i am only 3.4 smth
Now off to build engine that scrapes all the challenges and works with all of them.
