External datasets

Music Demixing Challenge ISMIR 2021
#1

Hello everybody!
I have a question for the organizers. Are we allowed to use other public music datasets?

1 Like
#2

Hi @idoraquel,

Yes, you can use an external dataset.
But remember to mark external_dataset_used as true in your repository.

There are two leaderboards – one for systems that were solely trained on the training part of MUSDB18HQ (“Leaderboard A”) and one for systems trained on any data (“Leaderboard B”).

(source)

And there are prizes for both the leaderboards. :wink:

LEADERBOARD A

  • :1st_place_medal: 1st: 3500 CHF
  • :2nd_place_medal: 2nd: 1000 CHF
  • :3rd_place_medal: 3rd: 500 CHF

LEADERBOARD B

  • :1st_place_medal: 1st: 3500 CHF
  • :2nd_place_medal: 2nd: 1000 CHF
  • :3rd_place_medal: 3rd: 500 CHF

(source)

2 Likes
#3

We’re allowed to submit to both, is that correct?
Say I have a system which has some initial performance with only MUSDB18-HQ (submitted to leaderboard A), and then I continue training the same model with additional training data and submit that to leaderboard B.

#4

Yes, you can submit for both the leaderboards.

Please make sure that you use the external_dataset_used configuration properly, and don’t end up submitting Leaderboard B’s entry as A’s or vice versa. :smiley:

#5

The current leaderboard seems to show all submissions regardless whether external datasets were used or not. Is there a way to view leaderboards A and B separately?

3 Likes
#6

Can I train model on a non free dataset or use such pretrained model?

#7

Can I train model on a non free dataset or use such pretrained model?

Yes, you can train your own models on your data (which doesn’t have to be freely available) - these systems are than part of leaderboard B.