Flow Where You Want – Guidance for Flow Models
5 days ago
- #generative models
- #inference-time controls
- #flow matching
- The tutorial demonstrates adding inference-time controls to pretrained flow-based generative models to perform tasks they weren't trained to do.
- Two types of guidance are applied: classifier guidance for generating specific digits and inpainting to fill in missing pixels.
- Guidance works by adding velocity corrections during sampling to steer the model toward desired outcomes.
- The tutorial explores guidance methods in both latent space and pixel space, including PnP-Flow for iterative projection.
- Classifier guidance uses an external classifier to enforce class compliance, while inpainting fills masked regions based on surrounding pixels.
- Latent-space guidance is faster as it avoids propagating gradients through the VAE decoder.
- PnP-Flow achieves guidance by adjusting latent positions directly through forward and backward projections.
- The methods are applicable to various flow models and control tasks, enabling flexible steering of generative flows.