- The core debate in AI is whether intelligence begins with language or with building internal world models that predict consequences of actions.
- Yann LeCun's JEPA (Joint Embedding Predictive Architecture) predicts embeddings in latent space instead of raw pixels or tokens, avoiding blurry averages from next-frame prediction.
- Representation collapse, where models output the same embedding for all inputs, is a key challenge; approaches like Barlow Twins use cross-correlation matrices to avoid it.
- World models enable agents to predict future states, plan over action sequences, and act robustly in messy real-world environments, contrasting with behavior-cloning in VLA models.
- JEPA has shown promise in vision and video tasks (e.g., V-JEPA), but long-horizon planning remains difficult and likely requires hierarchical abstraction.
- The near-term future probably involves combining JEPA for world-state representation with LLMs for language interfaces, not replacing one with the other.