Game over for pure LLMs. Even Rich Sutton has gotten off the bus
3 hours ago
- #AI
- #LLMs
- #Rich Sutton
- Rich Sutton, Turing Award winner, is known for his 2019 essay 'The Bitter Lesson,' advocating AI progress through scaling rather than hand engineering.
- Sutton's essay is influential but criticized as overstated and flawed by some, including the author.
- Recent comments by Sutton suggest a shift in his stance, aligning more with critiques of pure LLM scaling.
- Major AI thinkers, including Yann LeCun and Demis Hassabis, have come around to critiquing LLMs, with Sutton being the latest.
- The critique emphasizes the limitations of LLMs and the need for world models, reinforcement learning, and neurosymbolic approaches.
- Investment in exploring beyond pure LLM scaling is advocated to address current limitations.