The Unreasonable Effectiveness of LLMs in Mathematics
9 hours ago
- #unconscious-mind
- #mathematical-discovery
- #AI-reasoning
- AlphaProof, an AlphaZero-inspired agent from DeepMind, uses reinforcement learning and Lean (a formal proof language) to construct mathematical proofs, achieving a silver medal in the 2024 International Math Olympiad (IMO).
- OpenAI's raw LLM (like GPT-5.x) later achieved a gold medal in the 2025 IMO without specialized scaffolding, outperforming AlphaProof by enabling messy, high-level reasoning in natural language rather than rigorous step-by-step proof construction.
- Jacques Hadamard's framework for mathematical discovery includes Preparation (conscious focus), Incubation (unconscious search for solutions), Illumination (unconscious idea emergence), and Verification (conscious rigorous validation), highlighting the subconscious as key to discovery.
- AlphaProof aligns with the Preparation and Verification stages but misses Incubation and Illumination, limiting its creativity by focusing on incremental, logical steps, while OpenAI's LLM better approximates the subconscious through flexible language-based reasoning.
- Looped language models (e.g., ByteDance's Ouro) allow reasoning in embedding space before token output, mimicking subconscious thought more closely and improving reasoning performance across benchmarks without increasing memorization.
- The future research agenda involves synthesizing deep unconscious reasoning (via looped models) with robust verifiers (like Lean for math) to close Hadamard's full discovery cycle, applicable across fields like physics, biology, and social sciences.