My submissions for a few days have been timing out, so I tried to revert to my latest working submission(212701). However (214747), it still failed, but instead of saying inference failed, it said (evaluation timed out) after approximately (30-40 mins of product matching phase, which is weird because other evaluation time outs usually fail after the 10 min mark).
Additionally, I noticed that git-lfs is keep track of old files, so the repo size is 14GB while my working directory is much smaller.
@dipam I have wasted so many submissions trying to figure this out, and it’s getting too frustrating now. can you please help me figure this out?
Many thanks.
1 Like
I don’t know the reason of failures but I can advise you to not to include .git file in your Docker image (if you created your own). In my case removing .git file make all pipeline running smooth.
1 Like
@youssef_nader3
Instead of killing the run after 10 minutes, we let the run complete and then raise and error if the time was greater than 10 minutes. However there is a hard limit after which we kill the run, evaluation timed out indicates that time limit is crossed. This time limit is set to 45 minutes. In this case, it would mean your code didn’t complete in 45 minutes. Let me know if this helps, else I can try to look into why it might have hung.
1 Like
@bartosz_ludwiczuk thank you for the advice, and good luck in this last week!
@dipam my submissions are now hanging even worse, I’m only getting Build Packages And Env: Preparing the cluster for you 
. for hours, I tried ignoring the .git folder, and also recreated the repository, submissions: 214988 215028 215057 are all hanging since yesterday.
I would really appreciate if you can look into this issue.
1 Like
Got same issue, “Build Packages And Env” running already 10 hours
@dipam its # 215016 and # 215013
1 Like
Similar issue for me 
takes now 12 hrs in building image.
still waiting same, more than 23 hours
1 Like