Scoring Announcement: Public vs. Private

AIcrowd
#1

Hello Participants,

We’d like to provide clarity regarding the scoring mechanism in the challenges.

While all submissions run on the public & private dataset, only the public scores are displayed while challenge is live. However, as the competition concludes, you will have the opportunity to designate your top three submissions. Only the private scores of these selected submissions will be considered in calculating your final leaderboard position.

Furthermore, all private scores for all your submissions will be disclosed at the conclusion of the challenge.

Thank you for your participation, and we look forward to your final standings!

All the best,
Team AIcrowd

🚨 Important Updates for Round 2
🚨 Important Updates for Round 2
#2

Thanks for the clarification! I think this is quite fair and reasonable.

#3

Two questions:

  1. The designation of top three submissions is per team and not per participant right? Otherwise a team with multiple members would have a big advantage, as scores fluctuate widely and a best of 9 would be much more likely to win than a best of 3.

  2. The calculation of the private scores was already done at the time of submission, right? So we will not have to worry if a solution may fail on the private test set because of going over the 2sec limit per image.

#4

This response is specifically for the MosquitoAlert Challenge 2023.

  1. The three submissions are designated per team, not per individual. Teams will be provided with a form to select submissions after the challenge concludes.
  2. The scores for the entire private dataset are pre-calculated, so there’s no concern about potential failures.
1 Like