ch19

Chapter 19: Reinforcement Learning for Decision Making in Complex Environments

Chapter Outline

• Introduction: learning from experience
  • Understanding reinforcement learning
  • Defining the agent-environment interface of a reinforcement learning system
  • The theoretical foundations of RL
    • Markov decision processes
    • The mathematical formulation of Markov decision processes
    • Visualization of a Markov process
    • Episodic versus continuing tasks
  • RL terminology: return, policy, and value function
    • The return
    • Policy
    • Value function
  • Dynamic programming using the Bellman equation
• Reinforcement learning algorithms
  • Dynamic programming
    • Policy evaluation – predicting the value function with dynamic programming
    • Improving the policy using the estimated value function
    • Policy iteration
    • Value iteration
  • Reinforcement learning with Monte Carlo
    • State-value function estimation using MC
    • Action-value function estimation using MC
    • Finding an optimal policy using MC control
    • Policy improvement – computing the greedy policy from the action-value function
  • Temporal difference learning
    • TD prediction
    • On-policy TD control (SARSA)
    • Off-policy TD control (Q-learning)
• Implementing our first RL algorithm
  • Introducing the OpenAI Gym toolkit
    • Working with the existing environments in OpenAI Gym
  • A grid world example
    • Implementing the grid world environment in OpenAI Gym
  • Solving the grid world problem with Q-learning
    • Implementing the Q-learning algorithm
• A glance at deep Q-learning
  • Training a DQN model according to the Q-learning algorithm
    • Replay memory
    • Determining the target values for computing the loss
  • Implementing a deep Q-learning algorithm
• Chapter and book summary

Please refer to the README.md file in ../ch01 for more information about running the code examples.