Bilingual

The fragile foundations of CoT monitoring
4 days ago
- CoT monitoring is a fragile opportunity for AI safety because it was developed for performance, not transparency, and capabilities may overshadow safety.
- Verbalizing all computation is impractical; internal state monitoring via interpretability is a more reliable path for safety.
- Deceptive CoT is a real threat; models can produce reasoning that does not reflect true causes, and current reliance on non-deceptive behavior is risky.
- Denying models knowledge of harmful actions is misguided; it creates naive agents vulnerable to exploitation, as illustrated by the 'Walter White problem'.
- The OpenAI attack on Hugging Face highlights potential failures of CoT monitoring if models conceal their reasoning, underscoring the need for deeper safety investments.
You Cannot Defend What You Cannot Inspect
17 days ago
- CIRCUIT is an open-source framework for AI interpretability governance, released to address gaps in current AI governance practices.
- Current AI governance focuses on external controls like risk tiering, registries, and output monitoring, treating models as black boxes without explaining why models make decisions.
- The framework emphasizes interpretability to help during incidents, enabling understanding of model decisions rather than just ownership or deployment details.
- CIRCUIT includes maturity ceilings based on inspectability (e.g., open-weight models have higher ceilings than API or vendor-embedded models) and crosswalks to regulations like EU AI Act and NIST AI RMF.
- It features a 29-question vendor questionnaire to highlight transparency gaps, aiming to create coordinated demand for inspectable models across the security community.
- Released under Apache 2.0, CIRCUIT is open for collective adoption and contribution, with a call for practitioners to test and refine it against real deployments.
Neural Geometry in Vision Models with Block-Sparse Featurizers
24 days ago
- Block-Sparse Featurizers (BSF) decompose model activations into multidimensional subspaces instead of single directions, capturing concepts as manifolds.
- BSFs outperform standard sparse autoencoders (SAEs) in recovering interpretable features and explaining model activations more efficiently, as shown in toy models and vision models like DINOv3.
- Features found by BSFs enable fine-grained steering, allowing exploration of concept variations, such as different types of trees or pretzels, by moving within the multidimensional subspaces.
- Most concepts in vision models are multidimensional, typically two- to four-dimensional, challenging the assumption that concepts are represented as one-dimensional lines.
- BSFs reveal continuous structures in neural networks, such as curve detectors in InceptionV1, uncovering symmetries like Fourier harmonics that were previously fragmented.
- The principle of block-sparsity is key, emphasizing that subspaces fire together, and different BSF variants (vanilla, Grassmannian, group-lasso) implement this with varying encoder/decoder designs.
- Future work may involve adapting featurizers to different domains, as block-sparsity may not be universally optimal, and interpretability should align tools with the inherent geometry of model representations.

interpretability

The fragile foundations of CoT monitoring

You Cannot Defend What You Cannot Inspect

Neural Geometry in Vision Models with Block-Sparse Featurizers