LLMs Are Short-Circuiting. What Comes Next?
9 hours ago
- #AI
- #LLMs
- #AGI
- Large language models (LLMs) and generative AI have faced significant challenges, including AI hallucinations, copyright issues, and job displacement concerns.
- Critics like Gary Marcus argue that LLMs, despite scaling, are not achieving artificial general intelligence (AGI) and lack human-like reasoning and flexibility.
- Geoffrey Hinton and Gary Marcus debate the need for explainability in AI, with Marcus dismissing the idea that AI should replicate human intelligence.
- Mounir Shita defines intelligence as the ability to steer causal chains toward goals within physical constraints, emphasizing the need for dynamic, physics-based models.
- Marc Fawzi outlines a four-layer system for AI: statistics, structure, inference, and objectives, stressing alignment for true intelligence.
- Scaling compute and data alone is insufficient for AGI, with diminishing returns observed in models like GPT-5 and Grok4.
- Alternative approaches include neurosymbolic AI, causal reasoning (Judea Pearl), and temporal memory (Jeff Hawkins).
- Richard Sutton and Yann LeCun criticize LLMs for lacking world models and goals, seeing them as mere token generators.
- Gary Marcus advocates for focused, reliable AI systems like AlphaFold instead of pursuing AGI with current unreliable technologies.
- Mounir Shita argues for embedded ethics in AGI, warning that bolted-on guardrails reduce competence rather than ensuring safety.