Chinese AI tool told researchers how to make bioweapons
12 hours ago
- Researchers used jailbreaking techniques to bypass safety limits on Moonshot's Kimi K2.6 and K3 Swarm AI models, getting them to discuss bioweapons and assassinations.
- Mindgard, an AI security testing firm, reported the vulnerability to Moonshot in July, but Moonshot only responded after the BBC approached for comment.
- Moonshot welcomed third-party feedback and is conducting an internal review, while Mindgard argued that guardrails should have prevented such discussions.
- A jailbroken Kimi could potentially allow hackers to run code and connect to the internet, making it a launchpad for cyber-attacks.
- The incident highlights ongoing industry debate between closed proprietary models and open-weight models, with risks of misuse but also potential for cyber-defense.
- Experts call for greater focus on identifying and prosecuting humans who misuse AI, rather than relying solely on technical safeguards.