The evaluation result does not match my local testing

NeurIPS 2019 : MineRL Competition
#1

Hi all,

I have just submit a pretrain version for evaluation, yet the result indicates that it goes only 1 episode with 4 steps; however when I ran the evaluate_local.sh on my own machine the output is 5 episodes with each at least 5k steps. The code files are totally identical. Anyone have any ideas about this situation? Thx.

#2

I have a same problem.
Evaluation successful but only 1 episode with 4 steps.

#3

Hey, I just solved mine. The issue is about the Tensorflow version. By default the system will install TF2.0, while my code is running on TF1.13, and the thing unexpected is the system not reporting any error about it. HTH.

#4

My code is also running on TF 1.13.1.
But TF version 1.13.1 is specified on my environment.yml file.
I’m still confused about what went wrong.

#5

I have a same problem.
Evaluation successful but only 1 episode with 4 steps.
But I used pytorch, and I don’t know what really happened.

#6

As far as i can tell, it seems that 4-steps-1-episode thing happen whenever sth going wrong inside the main script, while the system won’t report any of it (which sucks indeed). So what I can do is deleting part of my codes one by one and submit it again and again, until finding out the critical part…

#7

Too bad.
Check my above replay, see if could help.

#8

Thanks a lot! But the new maximum number of submissions is 25. That is too costly …