🚨 Clarification: Which are valid submissions?

Airborne Object Tracking Challenge
#1

1. Airborne Detection and Tracking Benchmark

  • Any submission with HFAR > 0.5 will be published on the leaderboard, but will NOT be considered for ranking.

  • :white_check_mark: All the submissions that have HFAR <= 0.5 will be ranked based on EDR on the leaderboard.

2. Frame-level Airborne Object Detection Benchmark

  • Any submission with FPPI > 0.0005 will be published on the leaderboard, but will NOT be considered for ranking.

  • :white_check_mark: All the submissions that have FPPI <= 0.0005 will be ranked based on AFDR on the leaderboard.

Are you using SiamMOT baseline?

  • Identical SiamMOT models (with delta <= 1.5% in EDR or AFDR) will be disqualified from winning the prize.

  • An identical model is a model that uses the exact same code and config file provided with the baseline.

  • :white_check_mark: Submissions that cross this threshold will be eligible for winning the prize.

What if I have a valid submission, but my score is lower than that of SiamMOT?

  • Your submission will remain eligible for the leaderboard and will be ranked in all the valid submissions.
  • :white_check_mark: You can win the prizes for being in Top 3 even if your score is lower than SiamMOT baseline.

Displaying eligibility information on the leaderboard

  • We have started displaying notes for all the entries which may be there on the leaderboard, but are not ranked, or are not eligible, along with reason.
  • :white_check_mark: If your submissions is wrongly classified or if a wrong submission is selected for the leaderboard, please do reply to this thread or DM me.

Example:

image
image886×158 19.2 KB

Check out these other relevant threads :raised_hands:

:genie: Requesting feedback and suggestions
:rocket: FAQs and common mistakes while making a submission

1 Like
🎉 Challenge extended to 1st September! (and more updates ⏬)
#2
#3

Dear team,

I have just entered this competition and submitted the baseline with a weird modification
https://gitlab.aicrowd.com/moto/airborne-detection-starter-kit/-/commit/de4b643d2f103b69ebe8ec035c7c2835b428a629#87d4a46f64a93f55d84724384f443b0d79d90801_82_83
(submission - http://gitlab.aicrowd.com/moto/airborne-detection-starter-kit/issues/2)

Is it allowed ? I suppose not but just wanted to make sure about that.

Cheers,
M

#4

Hi @moto,

It isn’t allowed because your algorithm needs to be online based on rules i.e. future frames can’t be used to predict the outcome of the current frame.

1 Like
#5

@shivam: Many thanks for your quick reply.

Does it also mean using historical frames is accepted ?

#6

Yes, historical frames are allowed.