Submission taking too long to evaluate

Hi @alfarzan can you take a look at submission 113508 please? Seems to be taking an awful lot of time to evaluate, not sure why or what, if anything, I can do to get around this.


I noticed the same thing as well. My submission is taking over 45 minutes to evaluate, although on my machine it runs in less than 1 minute! I suspect the server is overloaded.

Hi @navarunj

Can you try re-submitting it? We’ve just added some optimisations to try and help the internal docker images set-up faster so if you could resubmit that might have fixed it.

Hi, I also have this problem today. My 5 last submissions have been evaluating for up to 2 hours now, without success. Any idea?

Hi @Zidmie

We’re investigating this, please resubmit your models if the evaluation is taking more than 30min.
Or alternatively message me on discord or here and I can give you your full traceback while we are solving this issue :slight_smile:

Hi i am having the same issue, my submission Submission #114297 is taking forever.

Same thing here, submission 114248 Submitted yesterday evening. Still stuck

Hi all

It should all be fixed now, please re-submit any models that are stuck and they should run fast.

Apologies for the delays :slight_smile:

1 Like

It’s ok for me now, thanks a lot!


Submission is slow again… id 114409

Looks like submissions are getting slow again…

Same for me. The two submissions I tried today (one is #114626) never end, there are still evaluating after one hour.

Experiencing the same issue.

Also experiencing this issue

Also experiencing same issue. Eventually the submission failed with the following error. Any idea how to resolve this problem?

e[91mERROR: Could not find a version that satisfies the requirement catboost.utils (from versions: none)
e[0me[91mERROR: No matching distribution found for catboost.utils
e[0me[91mWARNING: You are using pip version 20.1.1; however, version 20.3.3 is available.
You should consider upgrading via the ‘/usr/local/bin/python -m pip install --upgrade pip’ command.
e[0m2021-01-15 10:31:12.258 | INFO | main:unzip_submission:55 - Unzipping the submission file
2021-01-15 10:31:12.259 | INFO | main:check_submission_file_type:93 - Checking submission file type
2021-01-15 10:31:12.263 | SUCCESS | main:check_submission_file_type:99 - Validated submission file type
2021-01-15 10:31:12.303 | SUCCESS | main:unzip_submission:88 - Extracted submission file to /src/
2021-01-15 10:31:24.003 | INFO | main:remove_docker_image:238 - Image build failed, not removing old images

Hi @zeming_yu

Sorry to hear that :frowning_face:

Could you please provide a submission ID so that I can look deeper into what is going on?

I can see from your error that you have a line:

e[0me[91mERROR: No matching distribution found for catboost.utils

That suggests to me that inside your requirements.txt file you have listed catboost.utils which is not a python package on its own. You should only need to install catboost and the rest would come with it.

Can you remove that line from inside the ADDITIONAL_PACKAGES list and retry?

It works now. Thank you very much!