Submission taking too long to evaluate

Insurance pricing game
#1

Hi @alfarzan can you take a look at submission 113508 please? Seems to be taking an awful lot of time to evaluate, not sure why or what, if anything, I can do to get around this.

Thanks!

#2

I noticed the same thing as well. My submission is taking over 45 minutes to evaluate, although on my machine it runs in less than 1 minute! I suspect the server is overloaded.

#3

Hi @navarunj

Can you try re-submitting it? We’ve just added some optimisations to try and help the internal docker images set-up faster so if you could resubmit that might have fixed it.

#4

Hi, I also have this problem today. My 5 last submissions have been evaluating for up to 2 hours now, without success. Any idea?
Thanks

#5

Hi @Zidmie

We’re investigating this, please resubmit your models if the evaluation is taking more than 30min.
Or alternatively message me on discord or here and I can give you your full traceback while we are solving this issue :slight_smile:

#6

Hi i am having the same issue, my submission Submission #114297 is taking forever.

#7

Same thing here, submission 114248 Submitted yesterday evening. Still stuck

#8

Hi all

It should all be fixed now, please re-submit any models that are stuck and they should run fast.

Apologies for the delays :slight_smile:

1 Like
#9

It’s ok for me now, thanks a lot!

2 Likes
#10

Submission is slow again… id 114409

#11

Looks like submissions are getting slow again…

#12

Same for me. The two submissions I tried today (one is #114626) never end, there are still evaluating after one hour.

#13

Experiencing the same issue.

#14

Also experiencing this issue

#15

Also experiencing same issue. Eventually the submission failed with the following error. Any idea how to resolve this problem?

e[91mERROR: Could not find a version that satisfies the requirement catboost.utils (from versions: none)
e[0me[91mERROR: No matching distribution found for catboost.utils
e[0me[91mWARNING: You are using pip version 20.1.1; however, version 20.3.3 is available.
You should consider upgrading via the ‘/usr/local/bin/python -m pip install --upgrade pip’ command.
e[0m2021-01-15 10:31:12.258 | INFO | main:unzip_submission:55 - Unzipping the submission file
2021-01-15 10:31:12.259 | INFO | main:check_submission_file_type:93 - Checking submission file type
2021-01-15 10:31:12.263 | SUCCESS | main:check_submission_file_type:99 - Validated submission file type
2021-01-15 10:31:12.303 | SUCCESS | main:unzip_submission:88 - Extracted submission file to /src/submission.zip
2021-01-15 10:31:24.003 | INFO | main:remove_docker_image:238 - Image build failed, not removing old images

#16

Hi @zeming_yu

Sorry to hear that :frowning_face:

Could you please provide a submission ID so that I can look deeper into what is going on?

I can see from your error that you have a line:

e[0me[91mERROR: No matching distribution found for catboost.utils

That suggests to me that inside your requirements.txt file you have listed catboost.utils which is not a python package on its own. You should only need to install catboost and the rest would come with it.

Can you remove that line from inside the ADDITIONAL_PACKAGES list and retry?

#17

It works now. Thank you very much!