I hope you’ve been tweaking and preparing your models in preparation of the final market
Here you’ll find
The final leaderboard process
Timeline to announcing winners
Duplicate accounts and no-code submissions
The final leaderboard process
We’re very aware that some of your models may have some errors on the final leaderboard. We will ensure that no one will be excluded from this leaderboard due to bugs. But we need your help.
The process will be:
Please use the regular submission locking procedure as before to pick your final submission.
After the deadline, we will generate all the prices from your models (some will fail)
We will reach out individually for models that have failed
You will have to respond within 24 hours to help us debug the submission
By the end of the week of Monday March 8 we will have run the leaderboard and will have the preliminary results
The timeline to announcing the winners
Winners will be announced on March 21st in the Insurance Pricing Townhall! More details about this to follow on Monday.
We will be separately contacting the winners for further due diligence and prize distribution.
Duplicate accounts and no-code submissions
We’ve noticed 2 small things that are going against the spirit of this competition.
Same person submitting from multiple accounts
Submissions that have no training code in the provided fit_model function. Please see here for more details on that.
These can lead to disqualification from the competition.
(We won’t disqualify anyone without first notifying them and having a conversation, so no need to worry about that. )
Will we get to see the preliminary leaderboard posted in the leaderboard section (like a new weekly profit leaderboard)?
Will everyone have to use the pick submission to lock in their final model or will the most recent submission be used? If manually locking in is required, I wonder if we should expect fewer participants in the final evaluation.
Not until the reveal on March 21st! But we will be independently contacting some members at the top of the final leaderboard to prepare short presentations for the town hall. The idea is to have the full leaderboard revealed live so that we can have a lively discussion at the time.
We will use exactly the same logic as before. If you lock a submission this week, then we will use that. If you don’t lock one this week, we then use your most recent submissions. If you don’t play this week, we will use the one we used for the last leaderboard.
Regarding (2) if at all you’re worried or if there is any confusion then I’d suggest just getting in touch with us so we can make sure the correct one is used
That’s a long time to wait for results . I love to see the leaderboard earlier. Plus, it would give us time to do some correlation analysis and formulate good questions to ask the winners. Hearing from the top participants is enough motivation for me to attend the townhall.
Are you guys done with all the scoring? Got an idea, How about listing the top 30 or 50 members/teams without rankings and you could announce the winners on the 21st. The rest could be listed. If we do not get a call from you, we already know that we did not make it to the top 3 or 5.
Haha we are indeed guarding this like a national secret.
But other than the “surprise factor” the real reason we are not publishing this until the very last moment is because, as you know, sometimes models crash for entirely unintended reasons, some did on the final leaderboard as expected.
What we are spending most of our time on is to make sure all models that should have worked, do work. That means working alongside participants to make sure the corrections are fair and justified and that if there was a crash, this opportunity to debug is not used maliciously (i.e. to change the model). To be clear, no one has tried.
In addition to that, we have to ensure the integrity of the market . That means running tests to make sure no handful of models are ruining markets (e.g. by pricing negatively) and so on. We were already running a set of these tests each week, but for this final leaderboard we are being extra careful.
All will be revealed on the 21st!
In the meantime, you can use the bonus round as your virtual pop-corn while you wait
The final leaderboard process
Timeline to announcing winners
Duplicate accounts and no-code submissions
Insurance Pricing Townhall! More details about this to follow on Monday.
)
. I love to see the leaderboard earlier. Plus, it would give us time to do some correlation analysis and formulate good questions to ask the winners. Hearing from the top participants is enough motivation for me to attend the townhall.
. 
. That means running tests to make sure no handful of models are ruining markets (e.g. by pricing negatively) and so on. We were already running a set of these tests each week, but for this final leaderboard we are being extra careful.
!