I tried submitting a new model, and even though it works on test data from /shared_data/data/test_data_sample, it fails while evaluating.
I can see the error, but without detailed information I have no idea what is going on, I only get from my Issue:
Submission failed : list index out of range
which is not helpful at all.
It also says that error logs are not accessible for users, and that “One of the admins will provide you access to the relevant section of the log that you might require for debugging.”
Can I then get access to my log? I even resubmitted the model with debug: True flag in aicrowd.json but it didn’t change anything.
Also, is this the only way to debug evaluation errors? If so, I can’t fathom trying anything advanced and then requesting access and waiting for it each time I have to debug.