Tutorial Deep Reinforcement Learning to try with PyTorch

Unity Obstacle Tower Challenge
#1

I am trying to start with PyTorch to cooperate my previous source code with this competition. Tensorflow is great, but Pytorch is the open-source code chosen in my community. So far, I have read all documentation on website and able to identify a few sources to try such as

https://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html#sphx-glr-intermediate-reinforcement-q-learning-py
https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893 (Not open yet)

Any more recommendation? And would it be benefit if we have a tutorial to start with Pytorch for this competition?

#2

In my opinion, a good start would be to take an existing PPO, SAC or Rainbow DQN implementation.
The initial challenges would be to prepare the model’s input and especially the model’s output, which shall support Multi-Discrete actions. The ML-Agents toolkit solves this by creating so called action branches. I think it is called policy branches in their code, which implements PPO.

#3

Incremental PyTorch implementations of main algos:
RL-Adventure DQN / DDQN / Prioritized replay/ noisy networks/ distributional values/ Rainbow/ hierarchical RL
RL-Adventure-2 actor critic / proximal policy optimization / acer / ddpg / twin dueling ddpg / soft actor critic / generative adversarial imitation learning / HER

Good implementations of A2C/PPO/ACKTR: https://github.com/ikostrikov/pytorch-a2c-ppo-acktr

BTW The repo for the Udacity course is open source: https://github.com/udacity/deep-reinforcement-learning

2 Likes
#4

Hi all,

I have been working on a tutorial with Pytorch for a couple of days. However, the main issue is the set up with configuration on machine, python version and libraries given the context of competition. Please share if any of you have successfully used Pytorch for at least one submission.

Thanks!