Toward artificial general intelligence
A full recap and where to go next.
Go back to the definition from lesson 1: programs that learn by trial and error from feedback that is sequential, evaluative and sampled, using non-linear function approximation. You have now studied every word of it.
Map of the methods
| Family | Learns | Examples |
|---|---|---|
| Value-based | Q(s,a) | Q-learning, DQN, DDQN, Dueling, PER |
| Policy-based | π(a|s) directly | REINFORCE, VPG |
| Actor-critic | π and V/Q together | A2C, A3C, DDPG, TD3, SAC, PPO |
| Model-based | The environment's T and R | Dyna-Q, trajectory sampling |
| Derivative-free | Tune parameters without gradients | Evolution strategies, genetic algorithms |
Advanced topics to explore next
Model-based Deep RLLearn a model with a neural network, then plan
Inverse RLLearn the reward by watching experts
Transfer · Sim-to-realTrain in simulation, deploy on real robots
Multi-task · Meta learningOne agent, many tasks — "learning to learn"
Multi-agent RLMany agents cooperating or competing
Your own problemBuild an environment with Gymnasium and try every algorithm
Next steps
- Run every lesson's code and play with γ, α, ε, λ
- Build your own grid world and solve it with value iteration and Q-learning
- Try Stable-Baselines3 to use PPO / SAC in real projects
- #DeepRL
- #ReinforcementLearning
- #PythonສຳລັບຄົນLaos
- #AI
- #ບອກຕໍ່ຄວາມຮູ້