Alice's Adventures in a Differentiable Wonderland
10 months ago
- #Machine Learning
- #Differentiable Programming
- #Neural Networks
- Neural networks are widely used in various applications like large language models, speech transcription, molecular discovery, and robotics.
- Neural networks are compositions of differentiable primitives, and studying them involves learning differentiable programming.
- This primer introduces the basics of optimizing functions via automatic differentiation and common designs for handling sequences, graphs, texts, and audios.
- Key design techniques covered include convolutional, attentional, and recurrent blocks, bridging the gap between theory and code (PyTorch and JAX).
- The primer aims to enable readers to understand advanced models like large language models (LLMs) and multimodal architectures.