- The Linear Representation Hypothesis suggests that concepts like truth correspond to directions in an LLM's embedding space, enabling truth probes via classifiers.
- AI safety researchers use truth probes to detect truthfulness, and they work surprisingly well on simple cases, but they are not universal truth oracles.
- Gödel's incompleteness and Tarski's undefinability of truth show that no sufficiently expressive language can contain its own total truth predicate, leading to paradoxes.
- A diagonal attack constructs a self-referential sentence about the probe's output, creating a liar paradox that no fixed truth probe can resolve.
- Empirical tests on a simple truth probe for Qwen3.5-4B show high accuracy on regular sentences but nonsensical scores on diagonal attack sentences.
- Lawvere's fixed-point theorem explains that negation has no fixed point in {TRUE, FALSE}, but extending to [0,1] with continuous functions still fails to avoid all paradoxes.
- Truth probes are useful for understanding model behavior, but they cannot serve as definitive truth oracles, much like undecidability does not hinder mathematics.