Hello!
We are happy to announce the updated starter kit (v0.1.1) which includes an updated batch-prediction interface, optimnized inference using vLLM, updated docs for referencing huggingface models, using the search API, and more.
You can access the latest version of the starter kit here:
- AIcrowd / Challenges / Meta Comprehensive RAG Benchmark - KDD Cup 2025 / Meta Comprehensive RAG Benchmark Starter Kit · GitLab
Please note that this release might still contain some minor issues, and we appreciate your feedback. We will continue to refine the starter kit, so we recommend setting up your environment to easily rebase/incorporate future updates (e.g., using git pull --rebase origin main/aicrowd-starter-kit). Additionally, the changes to the Agent interface in this release of the starter kit are not backward compatible, so please ensure to update your local forks of the starter kit with the latest changes.
Additionally, based on updates from the Meta team, the search_api will undergo further improvements over the coming week, with a stable release anticipated soon.
The evaluators for Phase I are now live and accepting submissions. We wish you the best of luck!
We encourage you to follow the instructions in the starter kit to make your submissions. Post your queries or issues over here.
Thank you for your participation and valuable feedback!
Best regards,
AICrowd Team