Performance degradation after submission

#1

Hello.
If I test my algorithm in my computer with Obstacle tower 2.0 version, it goes to 6~7th floor on average.
However, if I submit my algorithm, it always cannot go over 5th floor and gets sum of reward as 6.8.
When I check the leaderboard, others also seem to have lower grades than round1.
Is the environment for evaluation different from github’s environment? (https://github.com/Unity-Technologies/obstacle-tower-env)
Thank you in advance.

#2

Hi @kyushik_min

There was an issue with the evaluation which should be fixed now. Please try again and let us know what results you get.

#3

Hi @arthurj
I just submitted my algorithm and it goes over 5th floor!
I think problem is solved. Thank you :smile:

#4

Also I think more complexity (additional layout, themes, etc) has been introduced in Round 2 so you can’t expect the agent to perform as well as Round 1.