Announcement - Round 1 Scores

NeurIPS 2019 : MineRL Competition
#1

Announcement - Round 1 Scores

We have reviewed multiple submissions that obtain rewards that should not be achievable in the MineRLObtainDiamond-v0 environment.

As this is due to an easily exploitable reward loop present in outdated minerl versions (prior to minerl 0.2.5,) we have decided to add 5 additional submissions to each team. The new maximum number of submissions is now 25.

Please verify submissions locally to ensure your current scoreboard results. Top submissions submitted using out-dated minerl versions (prior to minerl 0.2.5) will be re-run to verify their performance.

Additionally, participants should retrain their models to account for the reward loop removal.

On AIcrowd, the minerl version can be checked by looking for the minerl==<version> line in the requirements.txt file.

Locally, the python package can be updated with python -m pip install --upgrade minerl command.

2 Likes
#2

Thank you for your hard work.
I have a question about the leaderboard.
Is the re-run going to affect it, or is it going to keep showing the old, perhaps wrong, results?

4 Likes
#3

Thanks for update!
but，has the minerl updated to 0.2.6 ?

Collecting minerl==0.2.5
  ERROR: Could not find a version that satisfies the requirement minerl==0.2.5 (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.16, 0.1.17, 0.1.18, 0.2.0, 0.2.1, 0.2.2, 0.2.6)
ERROR: No matching distribution found for minerl==0.2.5
#4

Yes, there was an announcement on Discord.

#5

Thanks for your check.

#6

To clarify - we now have moved to Round 1.5! The scores of Round 1 will be for archival purposes only.

1 Like