Max duration limit exceeded

Food Recognition Challenge
#1

Hi! My submission failed with error “Generate Predictions On Validation Data: Max duration limit exceeded”. What does it mean? Does evaluation has time limit? I haven’t found any information about it.

#2

And also what is “Evaluate Predictions: No more retries left”?

#3

Hello @markpotanin

We started levying a timeout of 2000s for every 1000 images. Max duration limit exceeded essentially means the same.

The meaning of this error depends on the context. In your case, the evaluate predictions phase was failing with OOM. We run a try loop if the execution fails due to non-code error (in this case, it was failing due to OOM and not a code error).

#4

Ok, thanks! What GPU memory limit for evaluation?

#5

We use Tesla T4 (16 GB) during the generate predictions phase.

#6

Hi! Can you say please what’s wrong with my submissions v17 and v16?