Hello Participants,
We are excited to announce the winners of the Meta CRAG Challenge 2024. We appreciate your patience as we completed the due diligence process for the annotations.
We want to thank all participants for their efforts and contributions to improve RAG solutions. Over the last four months, the Meta CRAG Challenge has seen over 2000 participants from various countries, making over 5500 submissions. Below are the winners for each task. The final evaluation process can be found here: Final Evaluation Process & Team Scores
Task 1: Retrieval Summarization
-
Team db3 -
Team md_dh -
Team ElectricSheep -
Simple_w_condition Question: Team dummy_model -
Set Question: Team dummy_model
-
Comparison Question: Team dRAGonRAnGers
-
Aggregation Question: Team dummy_model
-
Multi-hop Question: Team bumblebee7
-
Post-processing Question: Team dRAGonRAnGers
-
False Premise Question: Team ETSLab
Task 2: Knowledge Graph and Web Retrieval
-
Team db3
-
Team APEX
-
Team md_dh
-
Simple_w_condition Question: Team ElectricSheep
-
Set Question: Team ElectricSheep
-
Comparison Question: Team dRAGonRAnGers
-
Aggregation Question: Team ElectricSheep
-
Multi-hop Question: Team ElectricSheep
-
Post-processing Question: Team ElectricSheep
-
False Premise Question: Team Future
Task 3: End-to-End Retrieval-Augmented Generation
-
Team db3
-
Team APEX
-
Team vslyu-team
-
Simple_w_condition Question: Team StarTeam
-
Set Question: Team md_dh
-
Comparison Question: Team dRAGonRAnGers
-
Aggregation Question: Team md_dh
-
Multi-hop Question: Team ETSLab
-
Post-processing Question: Team md_dh
-
False Premise Question: Team Riviera4
We welcome all teams to submit a technical report to the 2024 KDD Cup RAG Workshop. The report should follow the KDD submission format.
- Submission link: OpenReview
- Submission deadline: 11:59 UTC, Aug 2, 2024
- Submission format: Submissions are limited to 8 pages (excluding references), must be in PDF, and use the ACM Conference Proceeding template (two-column format). For more details, please refer to: KDD 2024 ADS Track Call for Papers
We extend our sincere gratitude to all participants who contributed to this event!
All the best,
Meta & AIcrowd