Submission format BirdCLEF 2020

LifeCLEF 2020 Bird - Monophone
#1

I had a look at a file, perfect_run.csv, for the submission format. I wonder what is the last column, in which all records were numbered by 1.0. I understand that each 5-second interval can have the maximum number of 10 predicted species. Do we need to rank these 10 according to some scores (confidence value) made by our approach or just treat them equally?

I guess the last column might be score since it is decimal, and in each interval, records are top species (max=10) “without” ranking. The evaluation measure (MAP) use only predicted species listed in each interval in the submission file, but does not use these scores/confidence values for evaluation.

PER49;00:05:10-00:05:15;hauthr1;1.0
PER49;00:05:10-00:05:15;horscr1;1.0
PER49;00:05:10-00:05:15;meapar;1.0
PER49;00:05:15-00:05:20;citwoo1;1.0
PER49;00:05:15-00:05:20;hauthr1;1.0
PER49;00:05:15-00:05:20;horscr1;1.0
PER49;00:05:15-00:05:20;strwoo2;1.0

Please confirm whether I correctly understand.
Cheers

#2

The last column is the confidence score which has to be a value between 0.0 and 1.0. The predicted species for a 5-second interval have to be sorted by descending score (but the eval script will do that for you).

So for your example it could look like:

PER49;00:05:10-00:05:15;hauthr1;0.88764
PER49;00:05:10-00:05:15;horscr1;0.2476
PER49;00:05:10-00:05:15;meapar;0.00325

In the perfect run file, I just set everything to 1.0. The official evaluation script will be a bit different than the one in the validation data since the test set contains files with no vocalization. Additionally, the cMAP metric uses the scores to assess class-wise performance of your approach.

Let me know if you have any other questions.

Thanks,
Stefan