"Trust Us" Is Not a Control Surface: Anthropic and the Case for Open Weights
12 hours ago
- #Open Source AI
- #AI Ethics
- #Regulatory Capture
- Anthropic's launch of Claude Fable 5 includes undisclosed routing for specific topics, visible downgrades to older models, and mandatory 30-day data retention for all prompts and outputs, even violating prior zero-retention agreements.
- The company secretly degrades performance for 'frontier AI development' without user notification, using methods like prompt modification, raising concerns about covert product sabotage and user distrust.
- Anthropic's CEO advocates for government-regulated testing and auditing of frontier AI models, funded by Anthropic, potentially creating regulatory capture that favors incumbent companies like itself.
- The U.S. Department of War designated Anthropic a supply chain risk after conflicts over usage restrictions, highlighting dangers of vendor control, even as Anthropic pushes for federal oversight of competitors.
- Historical parallels (e.g., printing monopolies, telegraph, FM radio) show that gating general-purpose technologies leads to stagnation, control, and societal harm, whereas open access fosters innovation and security.
- Open-weight AI models, like those from Meta or DeepSeek, offer auditability, privacy, and resistance to manipulation by allowing users to run and inspect models locally, preventing covert censorship and data retention issues.
- The author argues that distributed access to open models is crucial for societal advancement, preventing a dystopian future where a few companies control AI, surveil users, and suppress competition under the guise of safety.