Lesson 04 · Six core methods

Generative adversarial networks

Forger versus detective: two networks compete until both are good.

A GAN has two networks: a generator that turns noise into fake data, and a discriminator that tries to tell real from fake. They train in turns until the generator produces data the discriminator can't distinguish.

The example trains a GAN to produce points in 8 clusters on a circle and checks whether it covers all 8 — the test for mode collapse.

D: max E[log D(x)] + E[log(1 − D(G(z)))] G: max E[log D(G(z))] WGAN-GP critic: E[C(fake)] − E[C(real)] + λ·E[(‖∇C(x̂)‖ − 1)²]

Problem	Symptom	Fix
Mode collapse	Generates only a few kinds of output	WGAN-GP, minibatch tricks
Vanishing gradients	D gets too good, G stops learning	Wasserstein loss + gradient penalty
Want to choose the class	Samples come out mixed	Conditional GAN (CGAN)

Key takeaways

GANs don't model p(x); they learn to sample directly — hence sharp images
GAN training is all about balancing D and G

generative-ai-ep04-gan.py

import math, torch, torch.nn as nn

torch.manual_seed(0)
# Real data: 8 Gaussian blobs on a circle, a classic test for "mode collapse"
centers = torch.tensor([[math.cos(2 * math.pi * k / 8), math.sin(2 * math.pi * k / 8)] for k in range(8)]) * 2
def real_batch(n=256):
    return centers[torch.randint(0, 8, (n,))] + 0.05 * torch.randn(n, 2)

def mlp(out):
    return nn.Sequential(nn.Linear(2, 256), nn.LeakyReLU(0.2),
                         nn.Linear(256, 256), nn.LeakyReLU(0.2), nn.Linear(256, out))

G, D = mlp(2), mlp(1)                       # generator: noise -> point, discriminator: point -> real?
opt_g = torch.optim.Adam(G.parameters(), lr=1e-4, betas=(0.5, 0.9))
opt_d = torch.optim.Adam(D.parameters(), lr=1e-4, betas=(0.5, 0.9))
bce = nn.BCEWithLogitsLoss()

for step in range(4001):
    real, fake = real_batch(), G(torch.randn(256, 2))
    # 1) discriminator: real -> 1, fake -> 0
    d_loss = bce(D(real), torch.ones(256, 1)) + bce(D(fake.detach()), torch.zeros(256, 1))
    opt_d.zero_grad(); d_loss.backward(); opt_d.step()
    # 2) generator: fool D into saying 1
    g_loss = bce(D(fake), torch.ones(256, 1))
    opt_g.zero_grad(); g_loss.backward(); opt_g.step()
    if step % 1000 == 0:
        with torch.no_grad():
            dist, nearest = torch.cdist(G(torch.randn(4000, 2)), centers).min(1)
        hit = dist < 0.25                                  # sample landed on a real blob
        covered = (torch.bincount(nearest[hit], minlength=8) > 100).sum().item()
        print(f"step {step:4d}  D {d_loss.item():.2f}  G {g_loss.item():.2f}  "
              f"on target {hit.float().mean():.0%}  modes covered {covered}/8")

Output

step    0  D 1.41  G 0.75  on target 0%  modes covered 0/8
step 1000  D 1.26  G 0.79  on target 44%  modes covered 8/8
step 2000  D 1.16  G 0.92  on target 60%  modes covered 8/8
step 3000  D 1.19  G 0.92  on target 69%  modes covered 8/8
step 4000  D 1.21  G 1.00  on target 74%  modes covered 8/8

#GenerativeAI
#DeepLearning
#PythonສຳລັບຄົນLaos
#AI
#ບອກຕໍ່ຄວາມຮູ້

Main Menu

Change Language

Currency

Generative adversarial networks

Key takeaways