Acknowledging the outage of the evaluation server. And we are working on having that fixed, and will post an update here as soon as we have more information.
Hi, our system was healthy since last update but unfortunately not picking up newer submissions.
The issue has been addressed now but pending submission queue have lined up, the queue will be cleared shortly, you will continue to receive updates under Gitlab issues list.
Is there any way to know which tag corresponds to a given submission from the issue tab?
Now there are several queued but it is hard to know to which commit they correspond.
At the same time, it sounds like a good idea to provide this information directly on issue page content. We have added it as enhancement and will make it available in next release.
Acknowledging that the Image Building service has been baked. But all the submissions have been received, and we will requeue them manually and ensure that all submissions which were submitted in time do get evaluated.
Update: Some of the evaluations which were broken due to the clog in the image build service have been requeued. Confirming that the rest of the submissions are still safe in the queue, and will slowly be evaluated. Thank you for your patience.
For any specific queries regarding your submissions, you can also reach out to us at : mohanty@aicrowd.com (with devops@aicrowd.com) in cc.
@youngalou: Thats unusual o.O. We had checked that all the submissions during the outage were eventually evaluated (some of them failed because of unrelated errors in the code).
Can you point us to the relevant issues of the submissions which have not been evaluated ?