Hierarchical Reasoning Model – 1k training samples SoTA reasoning v/s CoT
9 months ago
- #AI
- #Machine Learning
- #Reasoning Models
- The Hierarchical Reasoning Model (HRM) is introduced as a novel recurrent architecture for AI reasoning tasks.
- HRM operates with two interdependent modules: a high-level module for abstract planning and a low-level module for detailed computations.
- With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using minimal training data (1000 samples).
- HRM outperforms larger models on the Abstraction and Reasoning Corpus (ARC), a benchmark for artificial general intelligence.
- Installation requires PyTorch, CUDA, and additional packages like FlashAttention for GPU compatibility.
- Training involves datasets for Sudoku, ARC, and maze-solving tasks, with specific commands for different GPU setups.
- Evaluation includes checking exact accuracy in Weights & Biases and using provided notebooks for detailed analysis.
- The model is documented in a 2025 arXiv paper titled 'Hierarchical Reasoning Model' by Guan Wang et al.