Going through the whole rules, I did not find any statements about whether we are allowed to use pre-trained models. In fact, it is easy to use the whole dataset (5000 (train)+10000 (purchased)+3000 (valid) ) locally and make several cheating pre-trained models.
If possible, please provide a statement of the allowance of the pre-trained models. If the dataset type is fixed during the competition, probably the best solution to prevent cheating is to forbid all pre-trained models and only allow learning from scratch since the dataset is small. Also, code review is needed to check the weird or complex weight initialization method.
@wufanyou : Only pre-trained models included in pytorch(via torchvision are allowed at the moment.
If you include any model weights in your submission, they will be scrubbed before the evaluation begins.
What if we use a string (e.g. base64 encode) to encode a model weight and put it into the .py file? Will this method will be treated as cheating? Based on this method, not only model weight can pass to the program, but also the image and label can be added to codes.
Clever ! But we are scrubbing all files in the submitted repo which are larger than a threshold size. And if you include a base64 encoded model which has the same filesize as any arbitrary python code file, that would be a great achievement
You could however split your base64 encoded model into many many little files, and even call them as .py files , but then your submission will be disqualified during the manual inspection of the code before any prizes are awarded.
We will however clarify in the challenge rules, that any pre-trained models (other than the ones included in torchvision) are not allowed. We will try our best to technically filter out submissions which disrespect the rule, and we will definitely disqualify any submissions which disrespect this rule during the manual code inspection, if by chance, some clever submissions fall through the cracks.
We would anyway appreciate, if participants respect the spirit of the competition
Thanks a lot. I wish to enjoy a competition that has no space for cheating. Then I can put more attention to the algorithm instead of fighting with those tricks.
In my past experience doing challenges in Aicrowd, the committee has been really fair. I spotted some accounts that I suspected of cheating that before I reported it (with evidence of course), they have already taken care of it.
@mohanty Thanks for the clarification! Does this mean we’re only limited to using torchvision? (I’m personally using keras applications which include the same models in torchvision, but I’m more comfortable with keras right now)
Sorry, what’s the right way to use pre-trained model?
I’ve tried “models.resnet18(pretrained=True)” but it has failed with
urllib.error.URLError: <urlopen error [Errno 99] Cannot assign requested address>
, but then your submission will be disqualified during the manual inspection of the code before any prizes are awarded.