Hello,
Attaching here the link to submission 327801’s report, and with it, the code and experiments repository below.
https://github.com/AndreiBulzan/arc-whitebox-submission-and-documentation/raw/refs/heads/main/report/ARC-White-Box-Phase1-Algorithmic-Contribution-327801.pdf
Thank you everyone for what was a very interesting (and fun) phase 1.
Really nice format! Intuitive and clear.
If I had an award for prettiest visuals, @andrei_bulzan would definitely be in the running.
Do you have any tips/tricks on how you built your vizes and the report itself? Is this a custom setup of yours?
FWIW, I, personally, very much appreciate efforts like these with a dedicated focus on the clarity of communication and quality of the final presentation.
Love the contribution @andrei_bulzan ! Thanks!
Also, a shoutout to @bgrubbs1984 's suggestion here, where he wanted to make visuals to:
explain complex problems in a simple (and entertaining) way that even a child can understand it
We need to celebrate more Mathematicians as Poets 
–
While we are in the process of reviewing all the write-ups we received for Phase 1, it’s clear many of them are principally LLM-written (which is fine, especially with a clear disclaimer), but I feel a vast majority of them would have benefited from just a few more conscious hours of genuine human touch: to tighten the narrative, improve clarity of communication, and really polish the final presentation. It would make the write-ups a lot easier to review for the reviewers, a lot more digestible for the community in general, and a delight to read for any reader.
I wonder if some individuals, who naturally enjoy that process, might want to collaborate more closely with others already writing the technical write-ups, to help improve them.
For the Phase 2 write-ups, we will try to do a better job of publicly sharing guidelines for the write-ups, but would also love to hear suggestions that the community has on what such guidelines should look like. (Very open to hearing from everyone, either privately over email at: arc-whestbench@aicrowd.com, or also publicly here on the forums.)
Disclaimer: I will personally not be involved in the review of the technical write-ups in consideration for the prizes. So opinions here my own. 
PS: For people interested in the topic, and unrelated to Whestbench, but this might be interesting:
Thanks a lot for sharing this!
If it is ok, I would like to share a few thoughts:
for write-ups, if I was an organizer I would have imposed a pages number cap. Ideally, max 10 pages (with standard characters). This would incentivize people into writing more of their own, and would make easier to review write ups. I myself had difficulty in reading reports of 20-50 pages (though there is a selection bias given these are the public ones, and I can not generalize on the rest).
for competition, I am not sure the current competition metric is helping solve the challenge itself. I am sure you and the other organizers thought deeply about it. But I think a less difficult setting, and cutting maximum flops might help steering people into tackling the core of the problem (though perhaps someone might try to go hard on training some prediction system). Or even making sampling based techniques too expensive. Phase 1 surfaced incentives that might have made competitors shy into pursuing more mechanistic approaches.
Last but not the least: it would be really, really appreciated if we had less LLM content on the forum itself!
Really appreciate the feedback, thank you.
To your question @james_rahenry, tips and tricks, absolutely, but no, there was no special custom setup behind these, or at least not besides what I describe below.
They vary in origin and technology, mostly what I use is html (three js for 3d, svgs - that svg pelican leaderboard’s not for nothing), ppt, and python generated plots*.
And they vary in the work or thought process behind them, I’ll give one example. Take page 66’s plot, which is just a sankey plot - styled a bit to the theme, but in essence, just a plot for which the compute numbers were input.
Now, take pages 3 through ~9-10. These came from this idea of ‘you’re just taking a gaussian distribution at input and passing its values through a network’. Which, sure, that’s nice and true, and maybe some people automatically click right away and can envision it, but I’d say a gaussian distribution can mean many things.
And to be able to make the leap to understanding, say, the importance of well distributed kerdock samples in 256d geometric space, you need to first have a way to convey the relationship between the numbers and geometric multidimensional representation first (visualizing a vector, understanding its length, direction, and how/why you can dispense of one).
If you can view the transformation (or perspective shift) from numbers to geometry, you will really care what these numbers are in detail, and thus, it becomes a prerequisite to understand that that Gaussian distribution is a bell curve centered in 0 with the bulk of its distribution being between -3 and +3, you understand how these numbers for which you draw vectors are selected, and you understand on what numbers ReLU gets applied.
I won’t linger for longer here, but trying to relay that here, for instance, that first gaussian drawing had this entire overarching idea that it had to serve, I wasn’t interested in drawing what a gaussian distribution looks like.
Now, I tried having this overarching idea (there may have been a few other ideas in the request) be given life by gpt 5.6, and, to no surprise, here’s an initial iteration of what came of it.
Threw it away whole, and then decided I need a way to view the geometry first, so I separated creating a simple vector visual, then iterated on how to emphasize the separation between length and unit direction (this was only roughly 2-3 prompts from this point, feels like it’d’ve been faster manually drawing the lines myself), while keeping the text that walks someone through the depiction; rinse and repeat for the other 8-9 pages that contributed to the representation of this idea.
For the visuals where you have an idea in mind, my tip is to describe it clearly in your own words. Do not use llm-written prompts, because slop begets more slop, and then iterate from it. The visual concepts and constructs are still a bit lacking behind in comparison to textual intelligence in the frontier models, and I think until a breakthrough or a few more model iterations come, they’ll still need a bit of babysitting and hand-holding - at least to get something deliberate and ‘premeditated’. The de-facto output from models will most often be something generic, but even through no other intervention than prompting and iterating, you can reach something aesthetically pleasing.
*maybe sub-tip, you can generate separate elements and/or generate visuals as part of a ppt presentation, then you will have more direct manual control and can intervene, move around, add your own shapes.
For the visuals where you don’t yet have an idea, ask either an llm for visual representations (in html, svg, ..), or a diffusion model and use them as inspiration to start from.
One thing I do, but not sure this is something I advise, is I discuss ideas and visual representations in text too. At some point the visual / mental representations blur, but I find that understanding something in text alone helps build the mental model that then can get translated into visuals.
Lots of words, hard to tell which you’ll find useful, but if I were to summarize: choose the level of detail and granularity that is appropriate for the task at hand. I think better men than me might be able to concoct some prompt (or mds, skills - I also avoid these, often find to degrade quality and tend toward generality, though I may not have given them sufficient time of day) capable of one shotting 10 or 80 pages at a time with visuals. Even one shotting 1 good visual alone I think would be wonderful, but I for one think some deliberate attention given to each visual in isolation is the best trick to getting it right. 1 at a time, and use the llm either in the driver seat if at a loss for visualization ideas, or as a handy tool that’ll do it with 1-2 (sometimes 10) good descriptions.
Also, review. Review and think if you’d be satisfied seeing said figure for the first time, and if the goal is to serve educative purposes, reread and see whether it achieves that (I still am unhappy with some from my report, but alas). What a model lacks in visual conceptualization skills, it partly makes up in automatic revision (text that goes outside of cells, overlapping content, etc. - these may slip, and may slip quite often, owing to pixel level inaccuracies, calculations and thinking in different embedding spaces between training and the canvas that it gets to work with now, but their vision module is sufficiently accurate to pick most of these up on review now and fix on run 2 or 3).
@mohanty
My 2 cents on submission format are that, 1. I loved the free form we had now, seeing everyone express their solution how they liked best. I think the majority went for journal like write-ups, some 5 pages with the very essence distilled, some 20-30 with details and visuals, some were normal report pdfs, etc. I liked it because freedom generally stirs creativity, but it’s fair that 2. that same creativity/freedom may make it harder to have a common ground to evaluate on, to know what to expect when going into a document, and to know what to evaluate by (you might have criteria that can fairly evaluate across all these formats, but I’m sure it’s not easy to compare with this much variety).
Unfortunately I have little to contribute beyond this reflection here, maybe just the idea that there’s a few dimensions across which you can confine the submissions (type of document, max num. of pages, subjects to tackle from the solution - e.g. focus on how it works? focus on what was the main 1 contributor to the solution? focus on the journey that got you to your best solution? ..), and that I think these decisions can be taken based on what the end goal is for the reader of said submission. E.g. Maybe you already have a strong idea on whether the submission should be self-contained and assume it comes with no other code or materials, or that the opposite is true, and that the submission needs to better unpack or reveal the decisions behind what already is publicly shared.
Whether the reader is a person (with sufficient time to read 5 pages? 50 pages?) or an llm (very likely not the case, but if so, favor tons of text), and what definition of practitioner should be considered - someone who simply has a decent understanding of the field and/or adjacent fields? or an actual competitor who’s probably aware of 80-90% of the ins and outs of the competition and problems it posed so far.
(now, again after having written these, I realize I say them all while also strongly believing in that point 1., where the guidelines set out so far say in large what they want to see in a report, and each team can decide what they deem a sufficient explanation of their algorithm, enough detail for an independent practitioner to reproduce, how deep to go into negative results, experiments and ablation studies).
And heh, not sure what adjective to use to describe the feeling, but it’s certainly something that I’m feeling thinking of the (in my opinion too, soon likely) world where we remain poets more than anything. A blend of anticipated nostalgia and excitement I think.
@Cipo, sorry to hijack, I know this last message wasn’t addressed to me, but only wanted to say on your ‘llms and natural language’ point that I, for instance, am biased in the other direction. I feel like I can tell cgpt and claude far better apart, almost right away, and also that fable and sol seem to have their way with more cryptic writing at times than any other models. Just sharing the perspective from my side, which I attribute to using (and consuming content written by) Gemini recently significantly less than others.
*Canva, blender and a few others too, though was not the case for the plots / visuals here.
