Lesson 01 · Introduction

What is generative modeling?

Learn p(x) from data, then create data that never existed.

A discriminative model answers "is this a cat?". A generative model learns what the whole dataset looks like — the distribution p(x) — and samples new examples from it.

The smallest possible example: learn the heights of 1,000 people as a normal distribution, "generate" 5 new people, and measure how plausible a height is.

DiscriminativeLearns p(y | x): data → label

GenerativeLearns p(x), then samples new x that look real

Six families of generative models

Approach	Family	Lesson
Approximate the density	VAE · Energy-based · Diffusion	3 · 7 · 8
Exact density, constrained model	Autoregressive · Normalizing flow	5 · 6
No density, just a sampler	GAN	4

Key takeaways

Goal: learn the data's p(x) and sample convincing new data
Likelihood tells you how plausible a data point is under the model

generative-ai-ep01-generative-modeling.py

import numpy as np

# A generative model learns p(x) from data, then samples new x from it.
rng = np.random.default_rng(0)
heights = rng.normal(170, 7, size=1000)          # "real" data we observe

mu, sigma = heights.mean(), heights.std()        # learn p(x) = N(mu, sigma^2)
new_people = rng.normal(mu, sigma, size=5)       # generate new samples

print(f"learned  mu={mu:.1f}  sigma={sigma:.1f}")
print("generated:", np.round(new_people, 1))
# Likelihood tells how plausible a value is under the model
for x in (170, 200):
    p = np.exp(-(x - mu) ** 2 / (2 * sigma ** 2)) / (sigma * np.sqrt(2 * np.pi))
    print(f"p({x}) = {p:.5f}")

Output

learned  mu=169.7  sigma=6.8
generated: [177.8 171.7 171.  171.5 160.3]
p(170) = 0.05828
p(200) = 0.00000

#GenerativeAI
#DeepLearning
#PythonສຳລັບຄົນLaos
#AI
#ບອກຕໍ່ຄວາມຮູ້

Main Menu

Change Language

Currency

What is generative modeling?

Six families of generative models

Key takeaways