High average reward but low score

quang_tran · June 20, 2020, 1:50pm

I’m a little bit confused. My mean reward is 8.134, yet my mean normalized reward is 0. I have 3 questions:

What are R_min and R_max used to calculate the normalized score for coinrun?
How many episodes are used to calculate the evaluation scores?
How are these evaluation episodes (levels) different from the episodes given in training?

jyotish · June 22, 2020, 8:06am

Hello @quang_tran

Something went wrong with the rollouts for two of your submissions. We are looking into it.

We are using these for R_min and R_max values

github.com

openai/train-procgen/blob/master/train_procgen/constants.py#L39-L56


EASY_GAME_RANGES = {
    'coinrun': [5, 10],
    'starpilot': [2.5, 64],
    'caveflyer': [3.5, 12],
    'dodgeball': [1.5, 19],
    'fruitbot': [-1.5, 32.4],
    'chaser': [.5, 13],
    'miner': [1.5, 13],
    'jumper': [1, 10],
    'leaper': [1.5, 10],
    'maze': [5, 10],
    'bigfish': [1, 40],
    'heist': [3.5, 10],
    'climber': [2, 12.6],
    'plunder': [4.5, 30],
    'ninja': [3.5, 10],
    'bossfight': [.5, 13],
}

The rollouts will be run for 1000 episodes with at most 1000 steps per episode. You can check the configuration used during rollouts here, Several questions about the competition