About final results and evaluation methodology

Hi Everyone,

You may have noticed the last few days of runs have not been updated on the leaderboard. This is intentional, as we will be announcing the final leaderboard at NeurIPS on December 10th, 2021.

One other important announcement is that we have made some changes to the real track evaluations. In an effort to give a fairer chance to all submissions and make them less affected by random artefacts near the real track. We re-ran all the submissions of Round 2 with manual interventions allowed per run. Specifically, each submission is allowed a maximum of 3 manual interventions per lap where we move the car back to the track at the same spot it leaves the track. This is repeated over 3 laps and the best scoring lap is selected as was done previously. All videos of the runs will be updated along with the final leaderboard which will be revealed at NeurIPS.

Thank you everyone for participating in the NeurIPS 2021 AWS Deepracer Challenge.

2 Likes