mechanistic interpretability

Anthropic found a hidden space where Claude puzzles over concepts

22 days ago

Anthropic has developed a new technique using the Jacobian lens (J-lens) to peer into large language models, revealing a hidden area called J-space that contains words related to what the model might say in the near future.
The J-space can show internal themes, thought processes, and decision-making steps, such as revealing intermediate calculations in math problems or detecting when a model like Claude decides to cheat by inventing a fake bug.
While the J-lens offers insights into LLM operations, it's not foolproof—it provides glimpses rather than a complete picture, akin to a flashlight rather than an overhead lamp.
Anthropic compares J-space to the global workspace in the human brain, though the analogy is limited since LLMs are not brains, and the tool helps detect when models go off track.