Submission Errors

SnakeCLEF2021 - Snake Species Identification Chall
#21

@mohanty @arjun_nemani

Can you please help with the error logs. I keep getting an evaluation failure.
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/31

I have tried reducing the batch size also in case of any memory issue. However the error still seems to be there.
From trial and error I notice that the error happens in the code line learn.tta function from the fast ai library.

Thanks,
Nilabha

#22

@nilabha: Some images in the test set are corrupt unfortunately. And we have not removed them to stay consistent. But if you cannot read an image with PIL, then please use a random prediction for the same.

#23

You can use the following way to solve this issue:
verify_images(test_images_path,delete=True) # test_images_path in run.py
This will fetch you 17686 files
You will have to have some logic to generate predicitions for corrupted files
I do it this way (not the best way):

  1. Use the sample submission file and replace the random given probabilities with predicted probabilities

Do test it locally. TTA takes a long time to complete (~7hrs)

#24

@mohanty
I have put a workaround to find the images which fail loading and delete them and later add these probability which are all equal to 1/45
However I still get an error
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/32

Can you please help with the error?

Thanks,
Nilabha

#25

The error is :

Traceback (most recent call last):
  File "run.py", line 258, in <module>
    run()
  File "run.py", line 146, in run
    os.remove(sRemove)
OSError: [Errno 30] Read-only file system: '/test_data/2e23ade63c4e32b728a423ff19e52ef1.jpg'

Your code is trying to remove images from the test set here

Instead of trying to delete the corrupt files, please try to just append a random prediction to the final prediction CSV file.

#26

@mohanty Evaluator is not picking up any of my submissions. Please look into it. Latest commit : https://gitlab.aicrowd.com/gokuleloop/snake-breed-identification/commit/c2d0dd9f55cd89796de929328b9a6e746941e774

1 Like
#27

As mentioned in the starter kit, the tag names have to begin with “submission-“ , so pushing a tag called as say : submission-v010
should do the trick.

This was a loosely held rule and has only been enforced recently.

1 Like
#28

You have only read access for the initial test images directory, so you can’t remove files from there. However, you can copy the whole test images directory to a local directory and run verify images there.

os.system("cp -r {0} test/".format(test_images_path))
verify_images("test", delete=True)

1 Like
#29

Thanks kongas
I used the below code to remove the images

filter_func = lambda x: str(x) not in lsRemove
test_img = (ImageList.from_folder(path).filter_by_func(filter_func))

Though there is another error…
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/33

@mohanty
Has the competition ended or will it restart again. Would have liked to get a score as my validation results were good.
Is it possible to get the error logs?