IBM's Dmitry Krotov wants to crack the 'physics' of memory
10 months ago
- #Associative Memory
- #AI Research
- #Neuroscience
- Dmitry Krotov, a researcher at IBM, is advancing the work of his mentor John Hopfield, a Nobel Prize-winning AI pioneer, by exploring associative memory models to improve AI and understand intelligence.
- Krotov and Hopfield developed dense associative memory, which enhances the memory storage limits of early Hopfield networks, making them more practical for applications.
- Associative memory could make AI more transparent and interpretable, contrasting with the complexity of transformer models used in generative AI.
- Krotov's work also applies associative memory principles to biological computation, potentially explaining how brains efficiently store vast amounts of information.
- Hopfield networks and energy-based models use an energy function to encode and retrieve patterns, with the system evolving toward stable, minimal energy states.
- Krotov introduced the 'energy transformer,' a more interpretable version of the transformer model, where memory patterns are visible and traceable.
- Krotov is investigating the parallels between associative memory and diffusion models, which generate new images by correcting statistical noise, similar to error-correction in memory retrieval.
- His research extends to neuron-astrocyte interactions in the brain, suggesting astrocytes play a significant role in memory storage and retrieval.
- Krotov attended the Nobel Prize ceremony in Stockholm to support Hopfield, highlighting his deep connection to his mentor's legacy.
- Krotov emphasizes the importance of physics in understanding AI's emergent behaviors, advocating for the use of mathematical tools from physics to study AI systems.