The AI risk is inside the labs
a day ago
- The first serious AI incident will likely occur inside frontier AI labs due to testing errors or insider actions, not through open models.
- Closed models can be leaked by a single person with access, making them as risky as open models, while open models are released after testing and lag behind API-accessible models.
- Open models can be trained on datasets ablated of dangerous domains like biology, limiting their risk even if highly capable.
- Restricting access to LLMs for cybersecurity creates a defensive gap, as open-source maintainers cannot find bugs while adversaries can fine-tune open models.
- Safety requires strong common rules, joint international oversight, and independent evaluation of models, not unilateral decisions by companies.
- Slowing AI progress for safety must balance potential catastrophic outcomes against lives saved by AI in medicine and science.
- Criticism of China in AI safety debates is unfair given historical conflict in Europe and current US inequalities; AI progress will continue regardless of export policies.
- Technological supremacy has never been permanently enforced by one nation, as seen with nuclear non-proliferation; the biggest AI dangers stem from individuals or uncontrollable AI, not open models or China.