The Implications of Linguistic Illegibility for LLM Security
2 hours ago
- Introduces 'linguistic illegibility' to describe scenarios where an LLM's external language outputs or mechanistic probes do not reflect its internal computation.
- Argues that linguistic illegibility is unavoidable because LLMs compute via math over activation spaces with lossy translations to/from natural language.
- Security mechanisms relying on linguistic self-reporting (e.g., chain-of-thought monitoring, constitutional self-critique, activation probing) cannot be completely sound.
- Proposes taint tracking as a promising sandboxing approach, defining system state that should never be influenced by model-produced data.
- Discusses additional sandboxing mechanisms like robust virtualization and third-party auditing to provide a foundation beneath linguistic monitoring.