A Safe Path to Open Weights
6 hours ago
- Safe open-weight models are public goods but carry real misuse risks; release must consider both the model's safety and the ecosystem's readiness.
- Thinking Machines released Inkling and Inkling-Small, two open-weight models, to democratize AI development and allow inspection of biases.
- Open-weight models can lower barriers for offensive cyber operations, but also enable defenders; staged releases and layered defenses mitigate risks.
- Inkling was assessed via internal evaluations, external red-teaming, and adversarial fine-tuning, showing no material incremental risk beyond existing open models.
- Research explores decoupling dangerous capabilities from general intelligence through pretraining data filtering, though it remains an open question.
- Safe release involves iterative stages (API access, fine-tuning APIs, open weights) to build ecosystem defenses and gather evidence.
- External collaboration (e.g., safety testers, defenders, researchers) is essential; Thinking Machines plans Tinker safety grants to support the ecosystem.