Artificial Cleverness: The system that knows everything and understands nothing

7 hours ago

AI systems sometimes make mistakes that seem trivial or nonsensical, such as "fixing" already correct code when not told that doing nothing is an option.
Research shows AI models rely on heuristics (rules of thumb) rather than true understanding; for example, Claude performs addition through memorized patterns and approximations, not step-by-step algorithms.
AI confabulates explanations: the process that generates answers is separate from the one that produces logical-sounding narratives, leading to a disconnect between what it does and what it says.
Terence Tao distinguishes artificial cleverness from artificial intelligence, comparing AI to a jumping machine that can't climb cumulatively, highlighting its lack of interactive, build-up reasoning.
AI excels in pattern-rich tasks (e.g., coding, summarization) due to dense heuristic coverage but fails in novel reasoning where relevant heuristics are absent, as shown by performance drops on unfamiliar problems.
The concept of AI as a 'heuristic companion' suggests it is a collection of sophisticated yet fragile heuristics, brilliant within its training distribution but unreliable outside it, akin to a Taylor series with a radius of convergence.
Uncertainty about whether heuristics can approximate true intelligence remains, but viewing AI as a heuristic companion helps practical use: leveraging its strengths for speed and context-rich tasks while recognizing its limits for novel problems.

Hasty Briefsbeta