Does the Bitter Lesson Have Limits?
9 months ago
- #AI
- #Computation
- #Organizational Theory
- The 'bitter lesson' emphasizes that general methods leveraging computation are most effective in the long run, despite short-term benefits of human-centric approaches.
- Rich Sutton's essay highlights historical examples like chess, Go, speech recognition, and computer vision where scaling computation led to breakthroughs.
- The bitter lesson aligns with broader blows to human ego, such as the Copernican Revolution and Darwinian thought, undermining human centrality.
- Ethan Mollick contrasts the bitter lesson with the 'Garbage Can Model' of organizations, where chaotic, undocumented processes make AI adoption difficult.
- Practical challenges to the bitter lesson include the need for high-quality, objective data and the difficulty of defining clear organizational objectives.
- Examples like Stockfish in chess show that combining human knowledge with efficient computation can outperform pure scaling approaches.
- Recent models like HRM demonstrate that smaller, task-specific models can achieve high performance without massive compute resources.
- The bitter lesson is a useful heuristic but must be balanced with practical constraints like cost, efficiency, and real-world applicability.