Generative Deep Learning
Teach computers to paint, write, compose and imagine: VAEs, GANs, diffusion, Transformers, all the way to DALL·E and Stable Diffusion. 14 lessons with PyTorch code that runs on an ordinary laptop.
- 14
- Lessons
- 13
- Sample code
- 3
- Parts
- Intermediate–advanced
- Level
About this series: A summary rewritten in our own words, following the chapter order of Generative Deep Learning (2nd edition) by David Foster (O'Reilly, 2023). All explanations and code are original and written in PyTorch, not copied from the book. Before running the code: pip install numpy torch scikit-learn gymnasium
Part 1 · Introduction
Part 2 · Six core methods
Variational autoencoders
Squeeze images into a latent space, then draw new ones from random points.
Read this lesson Lesson 04 · Six core methodsGenerative adversarial networks
Forger versus detective: two networks compete until both are good.
Read this lesson Lesson 05 · Six core methodsAutoregressive models: LSTM and PixelCNN
Generate text or images one token at a time.
Read this lesson Lesson 06 · Six core methodsNormalizing flows (RealNVP)
Invertible functions: turn noise into data and back again, exactly.
Read this lesson Lesson 07 · Six core methodsEnergy-based models
Score real data with low "energy", then walk downhill to create new samples.
Read this lesson Lesson 08 · Six core methodsDiffusion models
Add noise little by little, then learn to remove it — the heart of today's image AI.
Read this lessonPart 3 · Applications
Transformers and GPT
Attention: let every word decide which other words matter.
Read this lesson Lesson 10 · ApplicationsAdvanced GANs: StyleGAN and VQ-GAN
From ProGAN to VQ-GAN: the road to high-resolution images.
Read this lesson Lesson 11 · ApplicationsMusic generation
Notes are tokens too: predict the next note like the next word.
Read this lesson Lesson 12 · ApplicationsWorld models
Let the agent dream: learn the world as a model and train inside the dream.
Read this lesson Lesson 13 · ApplicationsMultimodal models: DALL·E 2, Imagen, Stable Diffusion
Type text, get an image: connecting language and vision.
Read this lesson Lesson 14 · ApplicationsConclusion: the future of generative AI
Recap the six families and where things are heading.
Read this lesson