Sutskever and LeCun: Scaling LLMs Won't Yield More Useful Results
14 days ago
- #AI
- #LLMs
- #Research
- Ilya Sutskever and Yann LeCun argue that large language models (LLMs) are hitting their limits, signaling a shift from an 'age of scaling' to an 'age of research'.
- Sutskever divides AI development into three phases: research (2012–2020), scaling (2020–2025), and a return to research (2025 onward), emphasizing the need for new ideas over just more GPUs.
- Current LLMs struggle with real-world usefulness despite strong benchmark performance, due to issues like hallucinations, brittle behavior, and poor generalization.
- Sutskever's new company, Safe Superintelligence Inc. (SSI), focuses on long-term research into superintelligence, new training methods, and safety, rather than consumer products.
- Yann LeCun criticizes LLMs for their shallow understanding of the physical world and advocates for 'world models' and architectures like JEPA, which learn from interaction with the environment.
- Both Sutskever and LeCun agree that scaling alone is insufficient for future AI progress, but differ on whether improvements will come from within the current paradigm or require a radical new approach.
- For developers and founders, the shift means focusing on use cases, data, and user experience rather than raw model size, and preparing for more diverse model types and hybrid stacks.
- The next decade of AI will be defined by research, new architectures, and how intelligence integrates into real human workflows, moving beyond 'predict the next token'.