Neural Thermodynamic Laws for Large Language Model Training
a year ago
- #Machine Learning
- #Thermodynamics
- #Large Language Models
- Introduction of Neural Thermodynamic Laws (NTL) for understanding LLM training dynamics.
- Theoretical demonstration of thermodynamic quantities and principles emerging under river-valley loss landscape assumptions.
- Practical guidelines for learning rate schedules derived from thermodynamic perspective.
- Mention of arXivLabs framework for community-driven development of arXiv features.