Label Noise Question

[ZFTurbo]

Will it be violation of rules if I automatically fix the label noise with some opensource algorithm/model (for example UMXL)? The question arises because this opensource model can be already trained on some “non-competition” data.

[subatomicseer]

Yeah, you cannot use models trained on other data as per this discussion:

… manual methods should be avoided as much as possible; you can still try to “clean” the data using some automatic method (as those can easily scale with the size of the dataset), but keep in mind that we only allow approaches that use the training data of the respective leaderboard (the corrupted one).
For example, you cannot apply a musical instrument tagger/classifier like YAMNet on the training set, as YAMNet has been trained with other data. …

Question on train dataset (Task A, B in Music Demixing Challenge)

Hi @jeonchangbin49 , thanks for your question. Your understanding of the three tasks (A, B, C) is correct: the objective is the same, the evaluation data is the same, but the training data differs. The reason for having leaderboads A and B is that we want researchers to investigate robust training methods, as issues like label swaps and bleeding happen very often when dealing with (big) training sets and they do impact the performance of the systems. For this reason, we generally prefer not …