Hello everyone,
We’ve got some great news to share, based on the valuable input we received during our recent office hours and on the discussion forum. Here’s what’s changing:
New Feature - query_time:
We’re adding a new parameter query_time to the generate_answer interface! This will be provided as a string. While we are maintaining backward compatibility with previous versions, we recommend updating your submissions repositories by rebasing with the starter kit repository.
Check out the details here.
Extended Timeout:
We’ve increased the timeout for per sample predictions to 30s. This change is temporary as we continue to benchmark various models on the competition specific hardware setup. Please be aware that we will adjust this timeout in the near future based on our findings.
Coming Soon - Batch Prediction Interface:
Stay tuned for our upcoming batch prediction interface, set to launch next week! This update is designed to enhance your control over GPU utilization in our multi-GPU setup. We’ll share more details soon.
Thank you for your ongoing feedback and contributions! Let’s continue to make great strides together.