[Final Week!] Leaderboard process, Winner announcement timeline, duplicate accounts

The final week is here!

I hope you’ve been tweaking and preparing your models in preparation of the final market :rocket:

Here you’ll find

:gear: The final leaderboard process

:trophy: Timeline to announcing winners

:warning: Duplicate accounts and no-code submissions


:gear: The final leaderboard process

We’re very aware that some of your models may have some errors on the final leaderboard. We will ensure that no one will be excluded from this leaderboard due to bugs. But we need your help.

The process will be:

  1. Please use the regular submission locking procedure as before to pick your final submission.
  2. After the deadline, we will generate all the prices from your models (some will fail)
  3. We will reach out individually for models that have failed
  4. You will have to respond within 24 hours to help us debug the submission
  5. By the end of the week of Monday March 8 we will have run the leaderboard and will have the preliminary results

:trophy: The timeline to announcing the winners

Winners will be announced on March 21st in the :classical_building: Insurance Pricing Townhall! More details about this to follow on Monday.

We will be separately contacting the winners for further due diligence and prize distribution.

:warning: Duplicate accounts and no-code submissions

We’ve noticed 2 small things that are going against the spirit of this competition.

  1. Same person submitting from multiple accounts
  2. Submissions that have no training code in the provided fit_model function. Please see here for more details on that.

These can lead to disqualification from the competition.

(We won’t disqualify anyone without first notifying them and having a conversation, so no need to worry about that. :heart:)

All the best and good luck!
Ali

8 Likes

Thanks Ali! Two quick questions:

  1. Will we get to see the preliminary leaderboard posted in the leaderboard section (like a new weekly profit leaderboard)?
  2. Will everyone have to use the pick submission to lock in their final model or will the most recent submission be used? If manually locking in is required, I wonder if we should expect fewer participants in the final evaluation.
1 Like

Hi @lolatu2

  1. Not until the reveal on March 21st! But we will be independently contacting some members at the top of the final leaderboard to prepare short presentations for the town hall. The idea is to have the full leaderboard revealed live so that we can have a lively discussion at the time.

  2. We will use exactly the same logic as before. If you lock a submission this week, then we will use that. If you don’t lock one this week, we then use your most recent submissions. If you don’t play this week, we will use the one we used for the last leaderboard.

Regarding (2) if at all you’re worried or if there is any confusion then I’d suggest just getting in touch with us so we can make sure the correct one is used :slight_smile:

4 Likes

That’s a long time to wait for results :stuck_out_tongue:. I love to see the leaderboard earlier. Plus, it would give us time to do some correlation analysis and formulate good questions to ask the winners. Hearing from the top participants is enough motivation for me to attend the townhall.

1 Like

Hi Ali,

Is the deadline for the final submission tonight 10 pm as usual?

I couldn’t find any information on this…

Thanks!

1 Like

No actually it’s midnight EST on Sunday! So you have a bit more time.

@lolatu2 yes ideally we would release it earlier, but I’m being conservative about it on purpose. If it is earlier we will announce it!

5 Likes

I don’t care about the $, I just want my mug! :slight_smile:

7 Likes

I just want my t-shirt! I’m not as interested in the mug, but I won’t say no to $$ :smiley:. @alfarzan What’s the process for getting the t-shirt and mug?

2 Likes

Hey @lolatu2!

Haha, we’ll be sending you both anyway, I guess. :smile:

I’ll reach out to you early this week and ship them.

2 Likes

Are you guys done with all the scoring? Got an idea, How about listing the top 30 or 50 members/teams without rankings and you could announce the winners on the 21st. The rest could be listed. If we do not get a call from you, we already know that we did not make it to the top 3 or 5.

Haha we are indeed guarding this like a national secret. :secret:

But other than the “surprise factor” the real reason we are not publishing this until the very last moment is because, as you know, sometimes models crash for entirely unintended reasons, some did on the final leaderboard as expected.

What we are spending most of our time on is to make sure all models that should have worked, do work. That means working alongside participants to make sure the corrections are fair and justified and that if there was a crash, this opportunity to debug is not used maliciously (i.e. to change the model). To be clear, no one has tried.

In addition to that, we have to ensure the integrity of the market :balance_scale: . That means running tests to make sure no handful of models are ruining markets (e.g. by pricing negatively) and so on. We were already running a set of these tests each week, but for this final leaderboard we are being extra careful.

All will be revealed on the 21st! :alarm_clock:

In the meantime, you can use the bonus round as your virtual pop-corn while you wait :popcorn:

2 Likes

So… It sounds like maybe the winners haven’t been determined yet? Can you reveal if any of the winners have been contacted?

Some people have been contacted to organise panelists and speakers for the townhall.

If these people also happen to be winners, they don’t know it yet because the leaderboard is not 100% final due to these checks :heavy_check_mark::ballot_box_with_check::white_check_mark:

1 Like

The real Lulz comes in when everyone realises that the @alfarzan account has been powered by a GPT-3 powered bot all along :laughing: !
Not giving any hints :laughing: :angel:

4 Likes

Hi,

Will tomorrow’s online event be recorded? I do not know if I will be able to connect in streaming, and I would like to see it later.

Thank you in advance!

Yes!

Even if we don’t release the full version online, we will be releasing an edited version.

1 Like