Hi All
I wanted to give a short introduction into the challenge to help newcomers have a start and evolve their algorithms from there.
To install Flatland follow there instructions here.
Once you have installed Flatland you can start playing around with the environment to get to know it and start developing your own algorithms.
The aim of the Flatland challenge is to find a controller that manages the traffic on any railway network as efficiently as possible, without creating any conflicts. You can achieve this by a number of different approaches.
- Optimization algorithms can be used to find optimal solutions for any given situation. However, you need to be aware that in Round 2 there will be stochastic events happening during an episode so you need to be able to re-schedule very quickly.
- You can use machine learning approaches (Reinforcement Learning is best suited here). For this we have written two basic introductions.
These introduction will give you a general feel on how to use predictors and observation builders and how to implement your own first DQN Controller.
Once you have understood what the observation builder and predictors do and you get a first feeling of why your agent can’t solve all the instances it is up to your creativity to improve on observation builders and predictors by building you custom ones.
Brief statement about the two:
- Predictor: The predictor is used because railway re-scheduling is not a local problem to be solved but rather needs some short term planning. To avoid conflicts you need to be able to anticipate the conflicts in advance. The stock predictor of flatland predicts that every train will take it’s shortest path at any time. This however will not be true, so it would be better to come up with ways to better predict, or learn to predict the future positions of the trains.
-Observation Builders: Flatland comes with three Stock Observation Builders that help you get an idea which features are relevant for a controller to solve the task. Most good performing controllers currently use the Tree Observation or modifications of it. Playing around with this observation you will quickly realize its limitations and that you need to enhance your Custom Observation Builder to include any relevant features to solve the task efficiently.
We suggest that you also try the Editor to debug your model and create problems for training your agents.
If you have any further question on how to get started don’t hesitate to reach out to the team. And we encourage you to have open discussions here in the forum about your ideas and problems.
The Flatland Team