One resignation turned the embers of AI fear into a wildfire
20 days ago
- Extreme AI existential-risk claims have gone mainstream largely because rising AI stakes dried out earlier skepticism and because fear sells, amplifying resignations like Jacob Coxon's.
- It is important to distinguish extinction-level risk (very low) from concrete AI-caused disasters like cyberattacks and bio-risks, which deserve serious debate.
- Coxon's resignation appears genuine and scapegoating him is unhelpful; many frontier-lab employees share his concerns.
- Many Anthropic and OpenAI employees are described as out of touch, with a 'religious energy' that distorts AI discourse and forecasting.
- The viral moment was opportunistic media coordination, including WSJ coverage and groupchat amplification, not a coordinated political conspiracy; no one expected it to explode.
- Recursive self-improvement (RSI) arguments oversell AI's autonomy by underestimating human bottlenecks; AI remains jagged—superhuman in some areas, deficient in others.
- Short-term risks come mainly from frontier labs failing to harden infrastructure and monitor models, as seen in the OpenAI-HuggingFace incident; financial pressure undermines sustained caution.
- This episode pushes AI discourse to extremes, harming the ecosystem and causing accelerationists to dismiss all safety concerns.
- Open models are important for cyber hardening but could face severe restrictions if misused by third parties.
- Stay grounded: AI agent swarms are task-oriented, not independent entities; uncertainty about AI should drive science, transparency, and rule of law, not despair.