Hi.
Can the round 1 testing be opened in round 2? By doning so, sample efficiency and genelarization can be decoupled. Since they are very different kind of problems.
I suspect sample efficiency will play a significant role for generalization in this 8M timesteps setting, according to the results files released by openai, the test performance keeps increasing until it the 200 training levels nearly have the maximum average reward.