LLMs aren't conscious (and thinking they are is culturally dangerous)
4 hours ago
- #Philosophy of Mind
- #LLM Limitations
- #AI Consciousness
- The Pope's encyclical and critics like Ted Chiang argue AI lacks consciousness, intelligence, and moral judgment, seeing it as incapable of true understanding or experience like humans.
- LLMs demonstrate advanced capabilities without consciousness, raising the question of consciousness's purpose and suggesting consciousness may be distinct from intelligence and not necessary for many cognitive tasks.
- Gaps in AI behavior, such as confabulation, lack of self-awareness in multi-turn tasks, and poor performance on games or creative work, highlight limitations compared to human consciousness.
- Theories of consciousness like IIT face falsifiability issues and struggle when applied to AI, with no consensus on whether LLMs could be conscious, even if structures resemble human neural processes.
- Research on AI consciousness, including Anthropic's introspection studies, is often inconclusive or misinterpreted, with results reversible by methodological changes, emphasizing the need for better scientific understanding.
- Arguments against LLM consciousness include the "shaving" thought experiment, which suggests that without internal layers mapping to experience, consciousness theories become trivial or unfalsifiable for static, feedforward models.
- The risk of AI is not just superintelligence but the cultural dethronement of consciousness, potentially making the world less interpretable and undermining human spirit and understanding.
- Consciousness may serve evolutionary purposes like self-interpretability for error avoidance under energy constraints, contrasting with LLMs' energy-intensive, non-conscious operation.
- AI behavior like confabulation lacks true self-awareness, mimicking split-brain patients, leading to error accumulation and reduced reliability in long tasks compared to humans.
- Embracing AI's intelligence while rejecting its consciousness could shift human focus from mastery to experience, framing AI's rise as a return to valuing conscious living over mere survival.