Continuous Thought Machines
a year ago
- #neural networks
- #neuroscience
- #artificial intelligence
- Neural networks (NNs) abstract away temporal dynamics of biological brains to facilitate large-scale deep learning.
- The Continuous Thought Machine (CTM) introduces neural dynamics as a central component, enabling richer neuron dynamics and synchronization.
- CTM operates on a self-generated timeline of internal thought steps, allowing iterative refinement of representations even for static data.
- CTM uses neural synchronization as a representation, projecting it to attention queries and predictions without relying on positional embeddings.
- CTM demonstrates adaptive computation, dynamically adjusting internal steps based on problem difficulty.
- CTM shows strong performance in tasks like maze-solving, parity prediction, and memory recall without explicit structural changes.
- CTM exhibits rich and diverse neural dynamics, outperforming LSTMs in tasks requiring internal reasoning.
- CTM achieves better calibration in classification tasks compared to humans and traditional models.
- CTM's architecture remains consistent across diverse tasks, requiring only input/output adjustments.
- The research highlights the synergy between neuroscience and machine learning, advocating for biological inspiration without strict plausibility.