第 06 课 · 六大核心方法

标准化流（RealNVP）

可逆函数：把噪声变成数据，还能精确地变回去。

标准化流是一种可逆的神经网络。如果映射 x → z 可逆且雅可比行列式容易计算，就能用变量替换公式精确计算 p(x)。

RealNVP 使用耦合层：保留一半输入，对另一半做缩放和平移。这样很容易求逆，对数行列式就是 s 的和。示例在双月牙数据上训练，并验证逆变换是精确的。

log p(x) = log p(z) + log |det ∂z/∂x| coupling: y₁ = x₁ y₂ = x₂ · exp(s(x₁)) + t(x₁) log|det| = Σ s(x₁) inverse: x₂ = (y₂ − t(y₁)) · exp(−s(y₁))

模型	思路
RealNVP	耦合层 + 每层交替掩码
GLOW	加入可逆的 1×1 卷积
FFJORD	基于常微分方程的连续时间流

要点

流模型既有精确似然，又能快速采样
代价是网络结构受到限制

generative-ai-ep06-normalizing-flows.py

import math, torch, torch.nn as nn
from sklearn.datasets import make_moons

torch.manual_seed(0)
X = torch.tensor(make_moons(2000, noise=0.05, random_state=0)[0], dtype=torch.float32)

class Coupling(nn.Module):
    """RealNVP coupling layer: keep one coordinate, scale+shift the other."""
    def __init__(self, flip):
        super().__init__()
        self.flip = flip
        self.net = nn.Sequential(nn.Linear(1, 64), nn.ReLU(), nn.Linear(64, 2))
    def forward(self, x):                     # data -> latent
        a, b = (x[:, 1:], x[:, :1]) if self.flip else (x[:, :1], x[:, 1:])
        s, t = self.net(a).chunk(2, 1); s = torch.tanh(s)
        b = b * torch.exp(s) + t
        out = torch.cat([b, a], 1) if self.flip else torch.cat([a, b], 1)
        return out, s.sum(1)                  # log|det J| is just the sum of s
    def inverse(self, z):                     # latent -> data (exact inverse)
        a, b = (z[:, 1:], z[:, :1]) if self.flip else (z[:, :1], z[:, 1:])
        s, t = self.net(a).chunk(2, 1); s = torch.tanh(s)
        b = (b - t) * torch.exp(-s)
        return torch.cat([b, a], 1) if self.flip else torch.cat([a, b], 1)

layers = nn.ModuleList([Coupling(i % 2 == 1) for i in range(6)])
opt = torch.optim.Adam(layers.parameters(), lr=1e-3)

def log_prob(x):
    logdet = 0
    for layer in layers:
        x, ld = layer(x); logdet = logdet + ld
    return (-0.5 * (x ** 2).sum(1) - math.log(2 * math.pi)) + logdet   # change of variables

for step in range(1500):
    loss = -log_prob(X).mean()
    opt.zero_grad(); loss.backward(); opt.step()
    if step % 500 == 0 or step == 1499:
        print(f"step {step:4d}  avg log-likelihood {-loss.item():.3f}")

with torch.no_grad():
    z = torch.randn(5, 2)
    x = z
    for layer in reversed(layers):
        x = layer.inverse(x)
    back = x
    for layer in layers:
        back, _ = layer(back)
print("generated points:", [[round(v, 2) for v in p] for p in x.tolist()])
print("invertible:", torch.allclose(back, z, atol=1e-4))

输出结果

step    0  avg log-likelihood -2.991
step  500  avg log-likelihood -0.358
step 1000  avg log-likelihood -0.306
step 1499  avg log-likelihood -0.286
generated points: [[0.91, -0.53], [1.45, -0.46], [-0.09, 0.26], [0.89, -0.53], [0.04, 1.01]]
invertible: True

#GenerativeAI
#DeepLearning
#PythonສຳລັບຄົນLaos
#AI
#ບອກຕໍ່ຄວາມຮູ້

主菜单

切换语言

货币

标准化流（RealNVP）

要点