You may have noticed a sudden drop in the public leaderboard.
The participant @tymur_prorochenko has let us know that he had used Round 2 data for some of his submissions. This is a difficult situation for us, as he points that not using the Round 2 data was not explicitly mentioned in the rules.
The organizers have decided to disqualify on the submissions that he said were using Round 2 data. Hence the change in the public leaderboard.
We will strive to avoid such data leakage future competitions, as providing logs always carries this risk.
Thank you for participating in the Citylearn Challenge, stay tuned for the final results.
Final leaderboard will be 20% Phase 1, 30% Phase 2 and 50% Phase 3, as announced earlier.
It will also use only the submissions selected in the form.
That said, we’ll investigate which other teams have used Phase 2 data for their submissions, it would be good if any such team is honest and comes forth with it now. (Please dm/email me if anyone wants to do so)
I suggest that the organizers should reproduce the process of training and submission of the competitors before announcing the ranking. In fact, it is also the player’s obligation to support and help the organizer with the process.