Deep Reinforcement Learning

For the unfamiliar: Reinforcement learning (RL) is a machine learning approach for teaching agents how to solve tasks by trial and error. Deep RL refers to the combination of RL with deep learning.

Table of Contents

Implementations

The notebooks helps through implementing various algorithms in reinforcement learning. All of the codes are in PyTorch (v0.4) or TensorFlow(v1.7) and Python 3.

• Dynamic Programming: Dynamic Programming algorithms such as Policy Evaluation, Policy Improvement, Policy Iteration, and Value Iteration.

  • Frozen Lake - v0

• Monte Carlo: Monte Carlo methods for prediction and control.

  • BlackJack-v0

• Temporal Difference: Temporal Difference Methods such as Q-learning, SARSA and Expected SARSA.

  • CliffWalking - v0

  • Taxi - v2

• Deep Q-Learning: Deep Q-Learning agents which use Fixed Q-Targets and Experience Replay for memory including improvements to Deep Q-Learning with state of the art Double DQN, Prioritized Experience Replay, Dueling DQN and Rainbow on range of various environmnts and using various deep learning libraries.

  • CartPole-v1 in TensorFlow.Keras

  • MountainCar-v0 in TensorFlow.Keras

  • LunarLander-v0 in PyTorch

  • Atari Pong in TensorFlow.Keras

  • Atari Breakout in TensorFlow.Keras

  • Self-Driving Car in PyTorch

Labs / Projects

The labs and projects can be found below. All of the projects use simulation environments from Unity ML-Agents. They are btter documented and requires good knowledge of the implemented algorithms to completely understand re-build.

Resources

• Cheatsheet: It is encouraged to use this PDF file to better understand the reinforcement learning algorithms.

Dependencies

To set up the python environment to run the code in this repository, follow the instructions below.

1. Create (and activate) a new environment with Python 3.6.

   • Linux or Mac:

   conda create --name drlnd python=3.6
   source activate drlnd

   • Windows:

   conda create --name drlnd python=3.6 
   activate drlnd

2. Follow the instructions in this repository to perform a minimal install of OpenAI gym.

   • Next, install the classic control environment group by following the instructions here.
   • Then, install the box2d environment group by following the instructions here.

3. Clone the repository (if you haven't already!), and navigate to the python/ folder. Then, install several dependencies.

git clone https://github.com/udacity/deep-reinforcement-learning.git
cd deep-reinforcement-learning/python
pip install .

4. Create an IPython kernel for the drlnd environment.

python -m ipykernel install --user --name drlnd --display-name "drlnd"

5. Before running code in a notebook, change the kernel to match the drlnd environment by using the drop-down Kernel menu.

Credits

• Spinning up in Deep RL an educational resource produced by OpenAI that makes it easier to learn about deep reinforcement learning (deep RL).

• Key Papers in Deep RL that I've comprehensively went throgh.

• David Silver's course on Reinforcement Learning.

• Siraj Raval Youtube Video Tutorials

• School of AI - Move 37 by Siraj Raval.

• UC Berkley and OpenAI's Deep RL Bootcamp.

• Udacity's Deep Reinforcement Learning Nanodegree program was that one course that helped me to get started with understanding the concepts and implementing the algorithms in Deep Reinforcement learning. This is that one course which I highly recommend and AWS Deep Racer Scholarship Program program as well.