aicrowd-bot @aicrowd-bot added waiting_in_queue label 6 hours ago
https://gitlab.aicrowd.com/ipv6/otc/issues/1
Monday as a deadline is kind of a bad choice, because debugging the submission process on a weekend does not sound feasible.
1 Like
Now the bot is not even picking up new submissions. Anyone else experiencing the same issues?
Acknowledging the outage of the evaluation server. And we are working on having that fixed, and will post an update here as soon as we have more information.
The evaluation system is up and running again. Sorry for the inconvenience.
It does not seem to work for us
yep, it does not work
"$ git push aicrowd submission-vXX" does not initiate a new evaluation
https://gitlab.aicrowd.com/ipv6/otc/tags
https://gitlab.aicrowd.com/ipv6/otc/issues
same issue over here
Hi, our system was healthy since last update but unfortunately not picking up newer submissions.
The issue has been addressed now but pending submission queue have lined up, the queue will be cleared shortly, you will continue to receive updates under Gitlab issues list.
Is there any way to know which tag corresponds to a given submission from the issue tab?
Now there are several queued but it is hard to know to which commit they correspond.
g
Hi,
You can view the commit id shown on issue page and match it with list of your tags.
For example:
https://gitlab.aicrowd.com/your-name/your-repo/tags
https://gitlab.aicrowd.com/your-name/your-repo/issues/1
At the same time, it sounds like a good idea to provide this information directly on issue page content. We have added it as enhancement and will make it available in next release.
2 Likes
I think it’s stuck again.
+1 on that. None of our submissions have been going through for the past couple hours.
+1
my last version not in QUEUE…
https://gitlab.aicrowd.com/wywarren/obstacle-tower-challenge/tags/v5.0
Upload at 14:30
Acknowledging that the Image Building service has been baked. But all the submissions have been received, and we will requeue them manually and ensure that all submissions which were submitted in time do get evaluated.
cc. @shivam
Update: Some of the evaluations which were broken due to the clog in the image build service have been requeued. Confirming that the rest of the submissions are still safe in the queue, and will slowly be evaluated. Thank you for your patience.
For any specific queries regarding your submissions, you can also reach out to us at : mohanty@aicrowd.com (with devops@aicrowd.com) in cc.
Will the requeued evaluations be reflected on the leaderboard? We still have the score of our 5th last submission.
@youngalou: Thats unusual o.O. We had checked that all the submissions during the outage were eventually evaluated (some of them failed because of unrelated errors in the code).
Can you point us to the relevant issues of the submissions which have not been evaluated ?
@mohanty Here is the latest submission:
https://gitlab.aicrowd.com/hermitcoder/obstacle_tower_by_42robolab/issues/95
It has been evaluated but the leaderboard does not reflect this evaluation.