as @ValAn told you, it’s better if you don’t change the defaults. But if you still need to change, make sure to just change the second parameter from the call to os.getenv.
This is because when you submit your code aicrowd expects you to “read” those paths from environment variables they’ve set.
For you to test that it works on your local machine it should be enough with the default values and uncompressing both test_metadata_small.tar.gz and test_images_small.tar.gz in the data folder. You can download both of those files in the resources page
I don’t use Keras nor Tensorflow, but if you are using conda - which you totally should (not just because it will make dependency management way easier but also because it’s a piece of software easy to use and which actually works straightforward) - it’s as easy as having your environment activated typing:
I apologize for overlooking this. Slow evaluation drove me crazy as I mentioned earlier in this discussion.
Now I wonder how I am supposed to know this?
Am I supposed to read through previous competitions to understand how to submit?
Also I really think you should add edit history for your challenge description? Two months ago I read it for this challenge and now I see it’s changed. Nothing important, you updated number of images which were originally just copy pasted from stage 2. I hope you will not take my comments as an offense, I am just trying to understand, share my experience and give some suggestions how to make it easier to participate.
Our sincere apologies for the inconveniences faced by you.
Regarding the slow evaluation speeds, given that we have to execute your code (and models etc) on a large number of test images, the evaluations are indeed slow. Your model has to make predictions for a large number of images. We are trying to improve this experience by providing better feedback in terms of progress etc, and will definitely address this in the upcoming version of the challenges.
Regarding the competition, we are providing all updates on this forum here, and we would be happy to answer any and all questions you have here. We are also working on better notification systems so that you get relevant updates from the challenge over emails and other notification channels on the platform that you subscribe to.
In the meantime, we really appreciate your feedback. Your feedback helps us make the platform much better for thousands of other users on the platform, and under no circumstances we take it as an offense.
(on behalf of the organizing team)
@amapic This is happening as these packages are only available for linux distribution, due to while installing them in windows (I assume you are using windows) is failing. This is unfortunately a limitation currently with conda.
In such scenario, I will recommend getting rid of above packages from environment.yaml and continue your conda env creation. These packages are often included being dependencies of “main” dependencies, conda should resolve similar package for your system automatically.
@devops@shivam what does the timeout mean? Anyone knows where I can find this information. I have asked this question numerous times after @devops commented my failed subs but they were ignored so I am bring it up here.
How am I supposed to debug Timeout? Some of my successful subs took longer to execute than most of those which failed because of timeout. I couldn’t come up with reasonable explanation for such behaviour. I hope you can help me to understand this.
How come some of my subs took 14h and didn’t fail if the limit is 8h? Then again, how am I supposed to know that timeout is set to 8h? Where is it written? I also thought for a moment that you keep changing the timeout limit? Can you confirm that this is not true?
inferencing time is way off. Locally my model on 1080ti takes ~10 minutes to execute so obviously it runs on CPU when submitted.
@ValAn No, I can confirm the timeouts haven’t been change b/w your previous and current runs. The only issue has been timeout wasn’t implemented properly in past and it can be reason why your previous (1 week old) submission get missed from timeout.
We can absolutely check why it is taking >8 hours instead of ~10 minutes on local. Can you help me with following:
The local run is with GPU? I can check if your code is utilising GPU (when allocated) or running only on CPU for whatsoever reason.
What are the number of images when you are doing locally? The server/test dataset have 32428 images to be exact, which may be causing higher time.
I think specs for online environment would help a bit in case there is significant difference from your local environment: 4 vCPUs, 16 GB memory, K80 GPU (when enabled)