An analytic theory of creativity in convolutional diffusion models
10 months ago
- #diffusion-models
- #machine-learning
- #creativity
- The paper presents an analytic theory of creativity in convolutional diffusion models.
- Identifies two inductive biases, locality and equivariance, that enable combinatorial creativity.
- Introduces local score (LS) and equivariant local score (ELS) machines for mechanistic interpretation.
- Demonstrates high predictive accuracy (median r² ~0.95) on datasets like CIFAR10 and CelebA.
- Reveals a patch mosaic mechanism where models mix local training patches to create novel images.
- Partially predicts outputs of self-attention UNets (median r² ~0.77), highlighting attention's role in semantic coherence.