Deep learning basics
Neural networks, losses and optimisers in 20 lines.
A neural network is layers of matrix multiplication alternating with activations (ReLU). Training adjusts the weights to reduce the loss using gradient descent and an optimiser such as Adam.
The example trains an MLP on 8×8 handwritten digits (bundled with scikit-learn, no download) and reaches about 90% accuracy.
y = softmax( W₂ · ReLU( W₁·x + b₁ ) + b₂ )
loss = −Σ y_true · log y
w ← w − lr · ∂loss/∂w
| Piece | Job |
|---|---|
| Dense / Linear | Matrix multiply + bias |
| Conv2d | Finds local patterns in images with small filters |
| BatchNorm · Dropout | Stable training, less overfitting |
| Adam | Optimiser that adapts the learning rate per weight |
Key takeaways
- Every generative model in this series is built from these pieces
- What changes from lesson to lesson is mostly the loss function
import torch, torch.nn as nn
from sklearn.datasets import load_digits
torch.manual_seed(0)
X, y = load_digits(return_X_y=True) # 1,797 images of 8x8 digits
X = torch.tensor(X / 16.0, dtype=torch.float32)
y = torch.tensor(y)
train, test = slice(0, 1400), slice(1400, None)
model = nn.Sequential(nn.Linear(64, 128), nn.ReLU(),
nn.Linear(128, 10)) # 10 classes
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
for epoch in range(60):
loss = nn.functional.cross_entropy(model(X[train]), y[train])
opt.zero_grad(); loss.backward(); opt.step()
if epoch % 20 == 19:
acc = (model(X[test]).argmax(1) == y[test]).float().mean()
print(f"epoch {epoch + 1:2d} loss {loss.item():.3f} test acc {acc:.2%}")Output
epoch 20 loss 0.158 test acc 87.66% epoch 40 loss 0.042 test acc 89.67% epoch 60 loss 0.020 test acc 90.93%
- #GenerativeAI
- #DeepLearning
- #PythonສຳລັບຄົນLaos
- #AI
- #ບອກຕໍ່ຄວາມຮູ້