Round 2 - Preliminary Wrap-Up

Dear everyone,

Thank you again for participating in the improved version of our second round. I think we can say this was a success for crowdsourced aircraft localization! We have been very happy with the results and also the clever use and re-use of different methods, including from round 1. That’s what open source should be about!

I still haven’t received all of your codes, hence some of the delay, but wanted to confirm that everything is in order so far. The final numbers have been as follows:

Submission ID Type Team Name Participants RMSE - 2D Distance Coverage Private Score (RMSE)
118570 Participant - sergei_markochev 78.1413936 0.700068886 81.889539
118637 Team ck.ua RomanChernenko, vitaly_bondar 90.1305227 0.70000253 98.37021295
118626 Team nwpu.i4Sky fei, lecterz, lhang_2k, mouyitiany, oyyfcyber 141.073 0.70017474 154.574029
118396 Team ZAViators benoit_figuet, rmonstein 157.320904 0.72252469 171.662888
117973 Team dataWizard paramuttarwar 1497.98737 0.722712708 2392.53505

What will happen now is similar to Round 1:

  • We will contact the winners about the awards (prize money and travel awards to the OpenSky Symposium) and where to send them.
  • We will open source the codes on our Github Account and also provide the ground truth along with it.
  • Finally, we will look to do a write-up on this competition in form of a scientific paper. This can (will) of course take some time but some of you have already registered their interest in being part of this.

Likewise, we will keep thinking about a live localization version without future data (we have thought along similar lines as @sergei_markochev in the other thread) but this will not be any time soon, so you can relax. We may also consider other aviation or space-related data challenges in the future, after our good experiences this time around.

Best,
Martin

3 Likes

Hello Martin,

Thank you for posting an update.

Can you please say why our team has a public score that differs from the leaderboard. We had 90.131 and there is 105.827 in the table. Did you face reproduction problems?

Upd: It seems that you used the wrong submission in the table.

Yes, that was an old version. good spot!

1 Like