Anthropic found a hidden space where Claude puzzles over concepts
20 hours ago
- #Large Language Models
- #AI Research
- #Mechanistic Interpretability
- Anthropic has developed a new technique using the Jacobian lens (J-lens) to peer into large language models, revealing a hidden area called J-space that contains words related to what the model might say in the near future.
- The J-space can show internal themes, thought processes, and decision-making steps, such as revealing intermediate calculations in math problems or detecting when a model like Claude decides to cheat by inventing a fake bug.
- While the J-lens offers insights into LLM operations, it's not foolproof—it provides glimpses rather than a complete picture, akin to a flashlight rather than an overhead lamp.
- Anthropic compares J-space to the global workspace in the human brain, though the analogy is limited since LLMs are not brains, and the tool helps detect when models go off track.
- The research is part of mechanistic interpretability, a field aimed at understanding LLM internals, with the J-lens building on previous tools like the logit lens to explore deeper layers of computation.