Why hasn't longer-horizon training slowed AI progress?
3 hours ago
- AI progress has not slowed as expected largely due to major efficiency gains per FLOP from fixing software bugs and other improvements.
- Humans have difficulty accurately measuring AI intelligence because it is easier to recognize when AI is less intelligent than when it is more intelligent.
- Capabilities of AI models depend on many traits beyond intelligence, such as persistence and working memory, which can be improved through various tricks rather than just more computing.
- AI development is currently dominated by unexpected breakthroughs and bug fixes rather than a simple scaling law, making progress unpredictable.