Deep Reinforcement Learning
Teach a computer to learn by trial and error, from MDPs to DQN, PPO and SAC. 13 lessons with formulas, tables and Python code that really runs.
- 13
- Lessons
- 10
- Sample code
- 5
- Parts
- Advanced
- Level
About this series: A summary rewritten in our own words, following the chapter order of Grokking Deep Reinforcement Learning by Miguel Morales (Manning, 2020). All explanations and code are original, not copied from the book. Before running the code: pip install numpy gymnasium torch
Part 1 · Foundations
Part 2 · Tabular RL
Balancing immediate and long-term goals
Policies, value functions and planning when the MDP is known.
Read this lesson Lesson 04 · Tabular RLBalancing exploration and exploitation
Many slot machines: which one do you play?
Read this lesson Lesson 05 · Tabular RLEvaluating agents' behaviors
TD and Monte Carlo: estimating a policy's value from experience.
Read this lesson Lesson 06 · Tabular RLImproving agents' behaviors
SARSA, Q-learning and double Q-learning.
Read this lesson Lesson 07 · Tabular RLAchieving goals more effectively and efficiently
SARSA(λ), Q(λ) and Dyna-Q: get more out of every sample.
Read this lessonPart 3 · Value-based deep RL
Introduction to value-based deep RL
When the Q-table is too big, a neural network takes over.
Read this lesson Lesson 09 · Value-based deep RLDQN and Double DQN
Making RL look more like supervised learning.
Read this lesson Lesson 10 · Value-based deep RLDueling DDQN and prioritized replay
An RL-aware network and replaying the experiences that matter.
Read this lesson