Hey @milva,
So, we have some beautiful imitation learning machinery, with the ability to generate and persist expert demonstrations from top OR submissions, and also with the ability to figure out expert demonstrations on-the-fly (ie no need to create an expert demonstration dataset, you can just compute the best action dynamically). And there’s also a script to convert all that to RLlib format so you can scale up training.
Sadly, though, all this went through multiple versions and is very poorly documented as of right now! We’re aware of it and will try to improve this aspect as soon as we can…
You can maybe get some help from here: Recreating Malfunctions
Also if you tell us more precisely what you are trying to do (pure IL with RLlib?) we may be able to nudge you in the right direction in the meantime.