In the run.sh, after calling the training script and saving the model, shall there be another call to an evaluation module?
Or would aicrowd_helpers.submit() take care of that?
Basically, or maybe none of these are necessary and run could simply not have any training or evaluation and these are probably handled on the server side? In which case, are we required to use certain naming conventions for pytjon scripts? Or the models should be probably saved in some path?
@amirabdi: No just the final aicrowd_helpers.submit() call should trigger the final evalution (which would happen in a separate trusted container).
You will just have to ensure that PYTHON_PATH contains the path of the location where aicrowd_helpers.py is located.
Thank you for the response @mohanty
How about including a call to the train_pytorch.py or train_tensorflow.py in the run.sh script? Is that necessary?
@amirabdi: The run.sh should include a call only to the train_* files.
But the evaluation is triggered by a aicrowd_helpers.submit() call inside the individual train_* files. So please do ensure that after your training is complete, models+representations dumped at the appropriate location, you do include the call to aicrowd_helpers.submit().