I tried re-submitting my agent from round #1 to see how it did on round #2 - but get the following error
did something change? (all i did was push a text change and re-tag my repro)
The following containers terminated prematurely. : agent
Please contact administrators, or refer to the execution logs.
Looking at the logs, it seems that you will need to upgrade the
ObstacleTowerEnv python api to version 2.0 for the submission.
Did the evaluation of the submissions change for round 2?
Like what reset parameters are used and how many random seeds?
The evaluation protocol is the same as from Round 1. So submissions are tested on five held-out seeds. We also use the same reset parameters from Round 1, with the exception of cases where there are now additions, such as the new additional themes on higher floors, or additional floor layouts.
@arthurj in other words, can we assume the ‘config=None’ for the evaluation environment?