Round 1 Completed

#1

Dear all,
Round 1 has been completed.
To compete in Round 2, the requirement was to be in the Top 20 with a valid submission.
Given that we have less than 20 teams on the leaderboard, I am glad to announce that all people who have submitted can now partecipate in Round 2! :slight_smile:

Final leaderboard for Round 1:
1) CIIRC-Incognite
2) mrrobot
3) robot-ea

On the other hand, we expected a wider partecipation (given that 158 people subscribed to the competition) so we will probably send a survey to all subscribers to understand why only a few subscriptions turned into submissions.

One of the main reason is probably that this challenge is very hard and complex to tackle!
There are many problems to face at once, such as:

  • Learning to properly abstract the environment
    i.e. recognizing objects in some way
  • Building models of the world
    i.e. learning how the state of the environment evolves by acting on it
  • Learning reliable skills
    i.e. learning appropriate actions that consistently achieve certain states
  • Planning
    i.e. how to reach extrinsic goals by applying the learned skills
    … and all these pieces of knowledge interact with each other and must be learned together.

Indeed so far the highest score (0.235) has been achieved by controllers which just stand still, since in many of the goals that we test it is advantageous to just leave objects as they are rather than pushing them around randomly.
An agent doing random movements scores about 0.100 instead.
I would like to make a mention here for teams AutoLearingMPI and isi who got their scores without resorting to these two baseline controllers.
isi team also got up to 0.134 in the latest submission (better than random), altought it was after the deadline had expired.

Moving on to Round 2!
So we have one month left to improve our controllers and beat both the random and the static controllers!

What are your expectation?
Will you beat the random and static controller?
What have been the main challenges for your team so far?

Round 2 will soon open.

Good luck and good to everyone for this last month of challenge!

#2

For our team, the task was challenging even with GT masks, and the best learned masks that we’ve got were obtained with pre-trained model (using ImageNet).
So given that in Round 2 both of this options are not available, and in addition, we would need to train on different hardware, we are not sure that it would be possible to fix all the problems in 2 weeks.
Maybe it would be reasonable to make Round 1 rules the final rules as the problem was already challenging enough (e.g. no one gets better than “Doing nothing” policy).
Also, it would be beneficial to know the prizes for winning places.