First of all, many apologies it took so long to get back to you. In hindsight, the deadline was likely a bit close to our 8th OpenSky Symposium, which required a lot of immediate attention. Apologies in particular to @paramuttarwar, who was the only one coming close to the 5000m cutoff for the public score. Unfortunately, it was a bit worse for the complete score. Still congratulations for being the best so far!
Here is the full leaderboard with all information:
|Rank||Type||Team Name||Participants||Submission ID||RMSE - 2D Distance||Coverage||Complete Score||Complete Score Coverage|
What happens now?
Since nobody has reached the 5000m limit required, we will extend this round! Contestants have another two months until January 31, 2021 to compete in this round.
The awards will remain the same, however, COVID-19 developments permitting an in-person event, the travel grant to our 9th OpenSky Symposium in Brussels in October/November 2021 is available again.
We will reduce the coverage requirement to 70% in order to make it easier on the participants to pick the right measurements and reduce the RMSE.
We will release a lot of material that should help interested contestants to improve their solutions:
a) The code of the five winning teams of round 1 is available on our Github: https://github.com/openskynetwork/aircraft-localization These codes are licensed under the GPL and can be freely adapted for round 2 solutions in our new rules (please reference anything that you use though, that is just good and decent practice!)
b) More discussion from one of the winning teams is available from their talk at the OpenSky Symposium: https://www.youtube.com/watch?v=msBtF0Swfn4
c) Their paper should hopefully be available with the proceedings soon providing even more detailed insights than code & video.
d) We will release a pre-print paper discussing the creation of the datasets, which will hopefully answer any more questions you may have!
As a last note, having spoken to some participants who felt the data was difficult and somehow artificially corrupted. That is (unfortunately!) not the case, this is real-world data that has to be dealt with in order to track aircraft using crowdsourced communications. That is why we run this challenge, to solve a difficult problem!