When the evaluation is carried out to some sample in Task2 Phase1, the problem of “Timeout while making a prediction” often occurs, resulting in an error validating model output. I wonder what could be causing this problem, and is there a possible way to avoid it?
Generally speaking, timeout means that your submission is too slow, and that you need to make your submission more efficient. For your information, we limit a maximum of 15 seconds per sample, and a total of 40 minutes in Track 2.
You can try the following techniques.
-
Install acceleration packages (e.g. Flash Attention).
-
Quantize your model (e.g. to 4bit)
-
Use smaller and more efficient models