Due Diligence

Dear @ashivani, @shivam, @vrv, @faroit and @StefanUhlich

I blended UMX-L with TFC-TDF-based U-Net and took the 5th place on the Leaderboard B. I received a message from @ashivani that I am eligible winner for the 3rd prize for the leaderboard B and I “should provide step by step instructions on how to train the pretrained models, so that we can replicate the score using the instructions on our end”. Deadline is 2021.08.09 11:59 PM UTC.

The UMX-L was trained on a private dataset. Should I request a tracklist from the authors and buy these tracks somewere to reproduce training? If not, how to describe this step in step-by-step instruction?

2 Likes

@_lyghter we can’t provide you with a tracklist. Since umxl was published under an open-source license, I don’t see a problem with reproducibility as long as you didn’t fine tuned the model

1 Like

@_lyghter oh and: congrats! :slight_smile:

1 Like

Thanks @faroit. Could you answer a few more questions about UMX-L?

I recently asked @ashivani about “How to reproduce the training” in the Readme:

(Q) There I plan to provide links to the original repositories because I used only pretrained models. It’s enough?

(A) No, you should provide step by step instructions on how to train the pretrained models, so that we can replicate the score using the instructions on our end.

Since I have to write something about training of pre-trained models, I plan to write like this:

python train.py --root {DATASET_PATH} --target bass
python train.py --root {DATASET_PATH} --target drums
python train.py --root {DATASET_PATH} --target other
python train.py --root {DATASET_PATH} --target vocals

Are models from UMX-L baseline were trained with such commands? How many GPUs you have used? How much time it took?

@_lyghter again, if you did not fine-tune the model using additional training data and managed to improve the scores due to blending I don’t see any reason why you would need to add info on training of umxl for the sake of reproducibility.

Still, your description is accurate, the umxl model was trained using the normal training scripts. umxl is double the hidden_size (1024) compared to umxhq. Other than that, we used the same number of gpus (one for each target), the training time was about 2 weeks on 2080 gpu.

1 Like

@faroit, my repository contains weights of UMX-L which are licensed for non-commercial use only (CC BY-NC-SA 4.0). Can I use MIT license for my repository? If not, which license should I choose?

@_lyghter if the model is modified or adapted (e.g. you created a new model based on umxl), the answer, unfortunately, is: no (see https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/)

1 Like

@faroit, thanks for the link

I don’t quite understand what it means: “model is modified or adapted”. I used the umxl weights “as is”, without fine-tuning. I just blended the vocals, extracted by umxl with the vocals, extracted by other model. Does it mean what the model was “modified or adapted”?

@faroit, I removed umxl weights from my repository and added instruction how to download it from Pytorch Hub. Can I release my repository under MIT License?

@_lyghter I am not a lawyer, so I can’t give you a legal advice. My understanding is that model code != weights. So, if you loaded the pre-trained umxl weights and used the results for a model fusion, this would be considered as modification of the weights itself. So, your code can be MIT as long as you did not export/redistribute the umlx weights as part of a joint model.

Ideally, your code would load the pretrained model from torch.hub directly.

as I understand, in the following Example it would not be allowed to redistribute fusioned_model under MIT license.

# load umxl weights CC4.0 SA licensed
model1 = torch.hub.load('sigsep/open-unmix-pytorch', 'umxl')
# load other MIT licensed model
model2 = torch.hub.load('user/other_model', 'other_model')
fusioned_model = torch.nn.Sequential([model1, model2])
torch.save(fusioned_model, ...)

however, you can of course create any MIT licensed code that uses model1 as part as your algorithm.

Sorry for the confusion, does this make sense?

1 Like

yes, see my answer above. you can do this

1 Like