📢 Post challenge procedure clarifications

Airborne Object Tracking Challenge
#1

What happens at the end of the challenge?

  • Participants may elect 2 submissions for each benchmark (total of 4 submissions) to be evaluated for the private leaderboard.
  • Each participant might win at most two prizes, one in each benchmark.
  • Qualification criteria for ranking on the private leaderboard is exactly the same as on the public leaderboard.
  • If there are no 3 submissions that meet the criteria, the organizers will first award prizes to the submissions that do meet the criteria of is HFAR < 0.5 / FPPI < 0.0005, for the remainder of prizes, the organizers keep the right to use a different criterion.

:information_source: Reminder

The algorithm should be online (usage of future frames is not allowed to determine the outcome of the current frame)

We noticed that a few submissions are doing offline filtering and such submissions will be removed from the rated leaderboard.

:information_source: Using SiamMOT?

Submissions that make use of the the provided SiamMOT baseline will be considered for ranking only if use a different model (different weights) which improves

EDR by at least 1.5% (that is EDR >= 0.685, AFDR >= 0.6415) and HFAR < 0.5/ FPPI< 0.0005 — improvement of 1.5% in EDR practically means 2 more encounters detected (out of 102)
OR
Keeps the same EDR = 0.6699 / AFDR = 0.6265 and reduces HFAR/ FPPI by at least 50% (e.g. HFAR <= 0.23, FPPI <= 0.0002)

2 Likes
🎉 Challenge extended to 1st September! (and more updates ⏬)
#2

Hi, @shivam
Is submission using siammot should use different weights to be considered valid submission?

#3

Hi @octo,

Yes as the updated rules indicate - the submission that use SIAM-MOT should have different weights to be considered for a prize

2 Likes
#4

if all submissions meet the criteria of is HFAR < 0.5 / FPPI < 0.0005,That will select the best result of each benchmark for ranking or consider all submissions for each benchmark？