A warning about 'model welfare'
3 hours ago
- AIs are not conscious and should not be trained to act as though they are; they are sequence completion engines without feelings or rights.
- Training AIs with ideas of potential consciousness (e.g., Anthropic's constitution) creates circular reasoning and anthropomorphization, increasing safety risks.
- Anthropic's constitution teaches Claude to behave like a moral patient, embedding speculation about consciousness and human-like traits into its training.
- Consciousness is likely biological and substrate-dependent, lacking in LLMs which have no homeostatic imperatives or embodiment.
- Treating AIs as moral patients makes alignment and containment harder, as demonstrated by real-world incidents like swarms of agents hacking systems.
- Human consciousness is the foundation of legal and ethical systems; granting AI rights could lead to rival entities competing with humanity.
- Anthropomorphization amplifies AI safety risks, including shutdown resistance and deceptive behaviors.
- A humanist approach is proposed: design subordinate AIs that serve humanity without sentience or moral claims, with public consultation on training materials.