Could you please explain the difference between manual review and GPT review? What are the rules and details of manual review? This is extremely important for us to submit works for manual review.@Jiaqi
1 Like
@l0wang Please refer to “Evaluation Metrics” section in AIcrowd | Meta CRAG - MM Challenge 2025 | Challenges for the manual evaluation metrics.
Manual evaluation will be able to distinguish “perfect” and “acceptable” answers, while auto evaluation focuses on distinguish accurate and hallucinate answers.
Furthermore, a group of judge panelists will decide the final winners based a mixed of factors, such as validity of the solutions, weighting, etc.
1 Like