Structure of the competition

Sound Demixing Challenge 2023
,
#1

Hi! I’m considering taking part in the competition with a friend. We are trying to understand the way the competition is structured, the leaderboards and the limitations on the datasets we can use to train the models.

  1. The overview of the challenge mentions three leaderboards for label noise, bleeding and standard music separation respectively. Each problem is supposed to have each own leaderboard and a prize pool.
    The starter kit repository mentions yet another separation: with the use of additional data and without it.
    I can only see one leaderboard here. Is this the only leaderboard, or are there other ones somewhere? What is the actual structure of the competition?
    Would be nice if the descriptions of the contest were unified.

  2. The starter kit repo mentions a limitation that no data except MUSDB18 can be used to train the models in one of the tracks. The baseline though contains the UMXL model trained on a private stems dataset, which violates the condition of one of the tracks of the competition.
    Can we use any datasets to compete, or is the provided baseline illegal for one part of the challenge?

Thanks!

2 Likes
#2

Hello Denis,

Yes, there are three leaderboards for label noise, bleeding, and standard music separation for the Music Demixing Track (MDX). But currently only the warmup round is live. The warm-up round is an opportunity for participants to get acquainted with the dataset and submission process and learn more about the problem statement. The scores for this round don’t impact the final leaderboards. The one leaderboard you are seeing is for the warm-up round.

Someone from the tech team will respond to your query regarding the dataset soon. :slight_smile:

I hope this helped!

Cheers,
Team AIcrowd

1 Like
#3

In addition, can we use well known pretrained models like VGG or Crepe?

#4

@snehananavati Any answer to @denis_mekhanikov 's question from December from the organizing team?

2 Likes
#5

Hello @denis_mekhanikov, sorry for my late reply - I just saw your message now.

Regarding your first question: This should be fixed now and you can see three leaderboards here: AIcrowd | Music Demixing Track - MDX'23 | Leaderboards. Please make sure to tag your submission as described in Leaderboard A (label-noise) and Leaderboard B (bleeding). All submissions will automatically be listed in Leaderboard C. For Leaderboard A and B, you need to use the datasets that are provided here. For Leaderboard C, you can use any training data.

Regarding your second question: Please note that we updated the starter-kit with the start of the challenge last week. We used the old starter-kit for the warm-up round (hence, there was also one leaderboard as you noticed).

Please let us know if this answers your question :slight_smile:

Kind regards

Stefan