Munich 1991: The Roots of the Current AI Boom
a day ago
- #Artificial Intelligence History
- #Deep Learning Foundations
- #Jürgen Schmidhuber
- The foundations of today's trillion-dollar AI industry were laid in 1991 by Jürgen Schmidhuber's lab at the Technical University of Munich.
- Key techniques published in 1991 include: the first Transformer variant (linear Transformer), unsupervised pre-training, neural network distillation, and deep residual learning.
- These innovations underpin modern large language models like ChatGPT, with deep residual learning being central to both LSTMs and ResNets.
- The lab also pioneered generative adversarial networks (GANs) for artificial curiosity, forming the basis of Generative AI.
- Schmidhuber's work influenced AI research, including recursive self-improvement and world models, and his 1991 contributions are cited in the two most-cited papers of all time.
- Despite Munich's early AI leadership, commercial AI has shifted to the Pacific Rim, raising questions about regional competitiveness.