[Announcement] Submissions for Round 1 now open!

william_guss · August 19, 2019, 8:27pm

NeurIPS 2019 Competition Submissions Open

We are so excited to announce that Round 1 of the MineRL NeurIPS 2019 Competition is now open for submissions! Our partners at AIcrowd just released their competition submission starter kit that you can find here.

Here’s how you submit in Round 1 :

Sign up to join the competition on the AIcrowd website.
Clone the AIcrowd starter template and start developing your submissions.
Submit an agent to the leaderboard:

Train your agents locally (or on Azure) in under 8,000,000 samples over 4 days . Participants should use hardware no more powerful than NG6v2 instances on Azure (6 CPU cores, 112 GiB RAM, 736 GiB SDD, and a NVIDIA P100 GPU.)
Push your repository to AIcrowd GitLab , which verifies that it can successfully be re-trained by the organizers at the end of Round 1 and then runs the test entrypoint to evaluate the trained agent’s performance!

Once the full evaluation of the uploaded model/code is done, the participant’s submission will appear on the leaderboard!

» Get started now! «

MineRL Competition Page - Main registration page & leaderboard.
Submission Starter Template - Starting code for submisions and guide to submit!
Example Baselines - A set of competition and non-competition baselines for minerl .

abiro · August 20, 2019, 2:51pm

Which version of Python will be available in the training environment?

seungjae_ryan_lee · August 21, 2019, 2:09am

Thanks for the great work!

It seems to me that the queue is stuck. Would you happen to know the reason why?

shivam · August 21, 2019, 5:38am

Hi, we use repo2docker in running submissions.

By default, repo2docker will assume you are using Python 3.6 unless you include the version of Python in your configuration files. repo2docker support is best with Python 2.7, 3.5, and 3.6.

You can learn more about specifying different packages and their versions from above link.

shivam · August 21, 2019, 5:43am

Hi @seungjaeryanlee,

Thanks for raising the issue. We had misconfiguration in our cluster auto-scaling. The issue has been fixed now and submissions should go through in parallel (instead of 1 at a time happening earlier).

abiro · August 21, 2019, 9:42am

Thanks! Which version of repo2docker are you using? The latest ones seems to support Python 3.7 which would be nice since I’ve been using that for development.

abiro · August 25, 2019, 7:11am

@shivam Could you provide a definite answer regarding Python 3.7 support please? I have to rewrite my code if it is not supported.

shivam · August 26, 2019, 11:14pm

Hi @abiro,

The documentation link which I shared above went wrong. It was pointing to earlier version of repo2docker (0.6) which have python 3.6 by default. We are using version 0.9.0 having python 3.7 by default (unless you specify otherwise in your runtime.txt).

Sorry for any confusion caused.

ermekaitygulov · September 7, 2019, 10:46am

Question about deadline of first round: https://www.aicrowd.com/challenges/neurips-2019-minerl-competition there is said that 1 round finishes in 48 days, but it differs from date in “important dates” (22 september). When first round finishes?

BrandonHoughton · September 9, 2019, 7:06pm

Apologies, we are still in the process of formally moving the deadline back. This required more funding to be able to offer additional travel grants to NeruIPS so the schedule can be pushed back, we should make the announcement tomorrow!