🚀 Code Submission Round Launched 🚀

[wac81]

@shivam
it‘s timeout for public test too？

have 2 process：

1. data process spend 160s
   ��███▋ | 240263/277044 [02:38<00:26, 1370.87it/s]
2. predict spend 27m
   [Predict] 2/271 […] - ETA: 27:15

i have no idea for timeout.

[mohanty]

@wac81 : Yes, the timeouts apply to both the Public and Private Test Phases. Also, we have increased the timeout to 120 mins - please refer to this post : 📆 Deadline Extension to 20th July && ⏳ Increased Timeout of 120 mins

Best,
Mohanty

[mohanty]

@wufanyou : The evaluation failed due to a timeout. The increased timeout of 120 minutes should fix this issue. We have re-queued your submission for re-evaluation.

Best,
Mohanty

[shivam]

Hi all, in case you feel your submission is running quite slow online v/s your local setup.
It might be a good idea to verify torch or relevant packages are installed properly.

Here is an example for torch:

How to install pytorch in your submissions? Can my pytorch use GPU?

Potential Issue: In case the challenge you are participating have GPUs, and your submission is running slower than expected online. One of the potential reason can be wrong torch version installed. Why is this a common occurance? It is undoubtedly the most famous package for the users, BUT have a special caveat. The PyPI releases for the PyTorch don’t have GPU releases, due to that it is easy to add torch in your requirements.txt and make the mistake. How to identify if you are using corre…

In case you are confused how to verify for your package, please let us know and we can release relevant FAQs.

[vitor_amancio_jerony]

Hi @shivam and @mohanty. To debug this, I did the following: I’m printing nvidia-smi on my prediction_setup method, right before loading my model and it seems it gets executed 2 times, and that’s why I’m getting this CUDA out of memory error.
The first time it loads correctly but it can’t load a second time without releasing GPU’s RAM.
Any idea why it’s loading 2x?

[shivam]

Hi @vitor_amancio_jerony, thanks for the logs.

We have identified the bug during the evaluation phase which caused the models to load twice, and is now fixed.
We have also restarted your latest submission and monitoring if any similar error happens to it.

[wac81]

i passed all process for task2

but i am not see my result on leaderboards, any issues?

here is submission:

AIcrowd Submission Received #194023 - initial-19

submission_hash : 2cbf126dab5610c3c6b108cfb7e40028e59e5106.

[mohanty]

Hi @wac81,

We do see this submission on the leaderboard of Task-2. Maybe you were checking the leaderboard of a different task ?

Best,
Mohanty

[wac81]

yeah，this is my second submission which is show up, and i can’t first submission.
but it’s ok.

[shivam]

Hi @wac81, the leaderboard contains the best submission from each participants (not all the submissions). I assume that may have caused the confusion.

You can see all your submission here (they are merged for all the tasks in this challenge, but you can identify them using relevant repository name):
https://www.aicrowd.com/challenges/esci-challenge-for-improving-product-search/submissions?my_submissions=true

Best,
Shivam

[caimanjing]

Hi, @shivam and @mohanty
my last two submissions failed and the debug logs disappeared (i.e. link to the log is not displayed on the page), but debug logs before them are available. Could you take a look at it?

submission_hash : [bbf64209f663e4dac1dbddde443a9a7495a836f1 ]( Files · bbf64209f663e4dac1dbddde443a9a7495a836f1 · caimanjing / task_1_query-product_ranking_code_starter_kit · GitLab (aicrowd.com)).

submission_hash : [efe0e19058c344b2b29e76387fd51497233d89d7 ]( Files · efe0e19058c344b2b29e76387fd51497233d89d7 · caimanjing / task_1_query-product_ranking_code_starter_kit · GitLab (aicrowd.com)).

[mohanty]

@caimanjing : You can find the error logs on the individual issues now.

[zhichao_feng]

Hi, @mohanty @shivam

I just made a submission, but I don’t know why it has the same code as the last one, and their submission hashs are completely the same. Could you please help me cancel the second submission with this hash? I don’t want it to take up my submission chance. The name is #194735 - 7.16.2.

submission_hash : 75fcd642d1b8659786cc66b7a77fa621d1fda1a0.

[dami]

hi @shivam and @mohanty, my submissions were failed and no logs were given again. All of them were stopped at 90 mins.
69047cec6f7f9fc2fc18a265895418972c8ae86b
527acda520e4d5221a8aa6b18574b0a50c6430ed
f18d54966cd711a22032d32ab09923264e235832

[shivam]

Hi @dami,

We have re-run these submissions along with the fix for your issue.
It should pass automatically or will show relevant logs in case of failure now on.

[zhichao_feng]

Hi, @mohanty @mohanty

My submission issue has not updated for a long time. Can you please take a look? Thank you.

submission_hash : 52602f246d1f15bc286ef4a49296df666d441560.

[mohanty]

@zhichao_feng : The submission is currently in the evaluation queue.

[xuange_cui]

During the last day, I had some submissions waiting in the queue for hours.
Could you provide more machine resources to ensure that our submissions start running as soon as possible？

Best，xuange

[K1-O]

Hi, @mohanty,

Our two submissions were labeled as failed, but these submission passed both public test and private test (also displayed as “graded” in submission page).
Is it actually failed or successfully evaluated?

submission_hash : ca0b6377179a5db5c1dfc8dd643231c305129d5f
d490d3d132583404e03bc25fe6b05dde7d6d9e85

[mohanty]

@K1-O : Those evaluations have succeeded indeed. The labels on the gitlab issue also have been corrected.
If the issue occurs again, please do not worry about it, if the status for your public_test_phase and private_test_phase are marked as Success 🎉, then it is correctly evaluated and is being counted towards the final leaderboard.

Best of luck,
Mohanty