Hi,
I’m wondering what does the split field mean? What’s the difference between the validation set and the public test set? I’ve noticed that they both have answers. Thanks!
Will the public test set used in the online evaluation of Task 1?
Hi shizueyy,
“split”: Data split indicator, where 0 is for validation and 1 is for the public test.
Please see here for more details.
Will the public test set used in the online evaluation of Task 1?
For Phase 1, yes. Phase 2 will use the private test set to select the final winners.
The CRAG Team
how would you decide the top n team if everyone use the exist answer to get the 0.98+ score?
1 Like
I’m also curious about this, lol. There are a lot of 0.98+ scores in task 1 now. @graceyx.yale Could you please help with this problem? How will you choose which teams can go to phase 2 with a lot of 0.98+ scores? Thanks!
what’s more, those teams have the same score!
Hello @can_yi @shizueyy @ni_kai_hua ,
Any team that has at least one successful submission in Phase 1 can go to Phase 2. And we will use a private test set (unreleased) in Phase 2 to determine the final winner.
Thanks,
The CRAG Team
Yes, that is right. 