扩散模型
一点点加噪声，再学会去除它——当今图像 AI 的核心。
前向过程在 T 步内不断给数据加噪声，直到只剩噪声。反向过程用网络（通常是 U-Net）预测所加的噪声，再一步步去除，最终得到新数据。
示例是一个二维的小型 DDPM：把噪声变成 1,000 个点，落在真实数据的双月牙形状上。
前向: xₜ = √ᾱₜ · x₀ + √(1−ᾱₜ) · ε ← 可直接跳到第 t 步
损失: ‖ ε − ε_θ(xₜ, t) ‖²
反向: xₜ₋₁ = (xₜ − βₜ/√(1−ᾱₜ) · ε_θ) / √αₜ + σₜ·z
噪声调度决定每一步加多少噪声（线性、余弦）
U-Net下采样块缩小、上采样块放大，并通过跳跃连接相连
DDIM确定性采样且步数更少，速度更快
要点
- 网络只学一件事：预测噪声
- Stable Diffusion、DALL·E 2 和 Imagen 都基于这一思想
import torch, torch.nn as nn
from sklearn.datasets import make_moons
torch.manual_seed(0)
X = torch.tensor(make_moons(4000, noise=0.05, random_state=0)[0], dtype=torch.float32)
T = 100
betas = torch.linspace(1e-4, 0.05, T) # noise schedule
alphas = 1 - betas
abar = torch.cumprod(alphas, 0)
# Noise predictor eps(x_t, t): a small MLP (a U-Net does this job for images)
net = nn.Sequential(nn.Linear(3, 128), nn.SiLU(), nn.Linear(128, 128), nn.SiLU(), nn.Linear(128, 2))
opt = torch.optim.Adam(net.parameters(), lr=2e-3)
for step in range(3000):
x0 = X[torch.randint(0, len(X), (256,))]
t = torch.randint(0, T, (256,))
eps = torch.randn_like(x0)
xt = abar[t].sqrt()[:, None] * x0 + (1 - abar[t]).sqrt()[:, None] * eps # forward: jump to step t
loss = ((net(torch.cat([xt, t[:, None] / T], 1)) - eps) ** 2).mean()
opt.zero_grad(); loss.backward(); opt.step()
print(f"final noise-prediction loss {loss.item():.3f}")
with torch.no_grad(): # reverse: denoise step by step
x = torch.randn(1000, 2)
for t in reversed(range(T)):
eps = net(torch.cat([x, torch.full((1000, 1), t / T)], 1))
x = (x - betas[t] / (1 - abar[t]).sqrt() * eps) / alphas[t].sqrt()
if t > 0:
x += betas[t].sqrt() * torch.randn_like(x)
dist = torch.cdist(x, X).min(1).values
print(f"generated 1000 points, median distance to real data: {dist.median():.3f}")
print(f"x range [{x[:, 0].min():.2f}, {x[:, 0].max():.2f}] (real data: [-1.0, 2.0])")输出结果
final noise-prediction loss 0.359 generated 1000 points, median distance to real data: 0.031 x range [-1.44, 2.29] (real data: [-1.0, 2.0])
- #GenerativeAI
- #DeepLearning
- #PythonສຳລັບຄົນLaos
- #AI
- #ບອກຕໍ່ຄວາມຮູ້