Submission Fail due to Private Test: Evaluation timed out 😢

Meta Comprehensive RAG Benchmark: KDD Cup 2
#1

@aicrowd_team

Could you help check what the reason is for the failure of those submissions:

Those submissions made small changes to previous successful submissions and all were tested successfully on the provided public dataset. Besides, all those submissions passed the Validation Step but got stuck during the start of the evaluations. So there are no progress bars for the evaluations.

Now I can idenfity which part in my code create the problem. But I am still not able to reproduce it offline. I wish I can get some error message from the log to help me solve the problem.

Best
Fanyou

#2

I meet the same question, have you solved it?

#3

I recently had this happen. My guess is that with the recent submissions, there is strain somewhere. My recently working commits no longer work. I looked at the debug log and saw that it took ~ 40 minutes to the git repo to get fully downloaded. I cleaned up my lfs objects and trying again, will update if it worked.