📢 Announcement: Addition of `query_time` to the `generate_answer` Interface, and increased Timeouts!

Hello everyone,

We’ve got some great news to share, based on the valuable input we received during our recent office hours and on the discussion forum. Here’s what’s changing:

:star2: New Feature - query_time:

We’re adding a new parameter query_time to the generate_answer interface! This will be provided as a string. While we are maintaining backward compatibility with previous versions, we recommend updating your submissions repositories by rebasing with the starter kit repository.

Check out the details here.

:star2: Extended Timeout:

We’ve increased the timeout for per sample predictions to 30s. This change is temporary as we continue to benchmark various models on the competition specific hardware setup. Please be aware that we will adjust this timeout in the near future based on our findings.

:star2: Coming Soon - Batch Prediction Interface:

Stay tuned for our upcoming batch prediction interface, set to launch next week! This update is designed to enhance your control over GPU utilization in our multi-GPU setup. We’ll share more details soon.

Thank you for your ongoing feedback and contributions! Let’s continue to make great strides together.

1 Like

What is the format of query_time?

I didn;t quite understand, how will the query_time parameter be used?

@nutansahoo : Please refer to this discussion on why query_time is a relevant parameter.

@jiazunchen : Please refer to the dev_data included in the starter kit, for the format of the query_time parameter.

The timeout for task 2 seems not to be extended now. This submission AIcrowd is killed after only 2400s in total. Will it be extended?
Moreover, is the total timeout 7620s now? Why my submission AIcrowd is killed after just 3000s???
Could you please provide an accurate description of the system?

@shizueyy : We apologize for the confusion. There was indeed a configuration issue at our end, where the per sample timeout was increased to 30s / sample, but the overall timeout was not proportionally increased.

This has been fixed now. And the two submissions you mentioned have been requeued for evaluation.