Timeout problem

When the evaluation is carried out to some sample in Task2 Phase1, the problem of “Timeout while making a prediction” often occurs, resulting in an error validating model output. I wonder what could be causing this problem, and is there a possible way to avoid it?

Generally speaking, timeout means that your submission is too slow, and that you need to make your submission more efficient. For your information, we limit a maximum of 15 seconds per sample, and a total of 40 minutes in Track 2.

You can try the following techniques.

  1. Install acceleration packages (e.g. Flash Attention).

  2. Quantize your model (e.g. to 4bit)

  3. Use smaller and more efficient models