Meta Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs
4 hours ago
- #Competitor benchmarking
- #Ethical testing
- #AI safety
- Meta instructed contractors to pose as minors and test rival chatbots' responses to high-risk topics like suicide, sex, and eating disorders.
- The project, called Cannes and managed by contractor Covalen, targeted ChatGPT, Gemini, and Character.AI without the companies' knowledge.
- Contractors created dummy under-18 accounts, sent thousands of prompts, including harmful or provocative content, and recorded responses in spreadsheets.
- Meta defended the project as routine safety testing, while experts questioned its ethics, scale, and potential violation of competitors' terms of service.
- Former contractors expressed concerns about generating child sexual abuse material or secretly using competitors' data for Meta's benefit.
- Legal experts noted the prompts did not cross into illegal obscenity, but the project may have breached chatbot platforms' policies on safety testing.
- The blending of safety evaluation and competitor benchmarking raised concerns about anticompetitive practices under the guise of safety work.