Tiny-diffusion: A minimal implementation of probabilistic diffusion models
a year ago
- #PyTorch
- #Diffusion Models
- #Machine Learning
- Minimal PyTorch implementation of probabilistic diffusion models for 2D datasets.
- Visualization of forward diffusion process on 2D points dataset.
- Reverse process illustration showing recovery of training data distribution.
- Ablation experiments on hyperparameters like learning rate and model size.
- Learning process sensitivity to learning rate.
- Model configuration struggles with line dataset, producing fuzzy corners.
- Longer diffusion process yields better output.
- Quadratic schedule not superior; suggests trying cosine or sigmoid.
- Model capacity not a bottleneck across hidden sizes and layers.
- Timestep information benefits model, encoding method less critical.
- Sinusoidal embeddings aid in learning high-frequency functions.
- References include Datasaurus Dozen, HuggingFace's diffusers, and others.