Why do invalid submissions count against quota?

Learning to Smell
#1

I can understand limiting submissions to reduce test set data leakage, but in the case of an invalid submission, there is no score returned. It would be helpful if these submissions did not count against the daily quota.

#2

Dear @robert_allaway,

This is a good point. I will see if this can be modified the rules to only consider valid submissions.

BR,

Guillaume

#3

@robert_allaway: Thanks for pointing this out, and we have fixed the same. Now upto 5 failed submissions will not count towards your submission quota. And we have also increased the submission quota to 10 per day.

Best of Luck,
Mohanty

2 Likes
#4

Thanks very much @guillaumegodin and @mohanty!!