Billion-Parameter Theories

4 days ago

Human history shifted from mystical explanations to scientific ones, with many natural phenomena being described by concise equations.
Complex systems, unlike complicated ones, involve dynamic interactions and emergent behaviors that resist decomposition and simple explanations.
Traditional scientific tools have struggled with complex problems like poverty, climate change, and mental health, often providing only partial solutions.
Modern AI and large language models offer a new approach to understanding complex systems by creating large, operable models that can simulate and predict behaviors.
The architecture of AI models (like transformers) might be compact and universal, while the trained models themselves are vast and domain-specific.
Mechanistic interpretability is emerging as a method to study how AI models represent complex systems, potentially uncovering new scientific insights.
This new approach suggests that many intractable problems may be solvable with sufficiently rich models, even if the answers are probabilistic rather than deterministic.

Hasty Briefsbeta