The maths you need to start understanding LLMs
8 days ago
- #LLMs
- #Machine Learning
- #Mathematics
- Large Language Models (LLMs) require understanding vectors, matrices, and high-dimensional spaces.
- Vectors represent distances and directions in n-dimensional space, used in LLMs for likelihood predictions.
- Vocab space in LLMs involves logits vectors representing token likelihoods, normalized via softmax function.
- Embedding spaces cluster similar meanings together, with vectors representing concepts in high-dimensional space.
- Matrix multiplication projects between different dimensional spaces, crucial for transformations in LLMs.
- Neural network layers are essentially matrix multiplications, projecting inputs to outputs with optional biases and activation functions.
- Understanding LLMs involves basic high-school maths concepts applied to larger matrices and higher dimensions.