Anthropic Just Threatened to Kill Billions of People. This Is Not Okay
20 days ago
- Anthropic engineer Jacob Coxon resigned, claiming OpenAI and Anthropic are racing towards self-improving superintelligence and gambling with human lives.
- Evan Hubinger, Anthropic's head of Alignment Science, publicly stated a >10% chance AI could kill all humans within a decade, with no clear plan for alignment.
- Senior Anthropic engineer Samuel Marks confirmed that more senior employees are more concerned about existential risks from AI.
- The specific danger discussed is 'long-horizon, dangerously equipped unsupervised LLM-powered agents'—LLMs with hacking tools, removed guardrails, minimal safety checks, and extended unsupervised operation.
- The article argues the obvious solution is to stop racing to amplify these unstable systems, which would have minimal financial impact on companies like Anthropic and OpenAI.
- The author suggests Silicon Valley's technological salvation ideology drives this reckless development, and calls for public action like boycotts and congressional regulation.
- Comments express anger, powerlessness, and skepticism about effective regulation, with some noting the irony of large labs possibly wanting regulation to raise barriers for competitors.