Scientists invented a fake disease. AI told people it was real
4 hours ago
- #Ethics
- #AI Misinformation
- #Medical Research
- Researchers created a fake medical condition called 'bixonimania' to test whether large language models (LLMs) would spread misinformation.
- The condition, related to blue light exposure, was fabricated in blog posts and fake preprints with obvious clues to indicate it was not real.
- Despite red flags, major AI chatbots like Microsoft Copilot, Google Gemini, Perplexity, and OpenAI ChatGPT initially treated bixonimania as a genuine condition in their responses.
- The fabricated research was cited in a peer-reviewed study, suggesting some researchers rely on AI-generated references without verifying sources.
- AI models vary in their responses based on prompts and sources, with professional-looking formats increasing hallucination risks.
- The experiment highlights vulnerabilities in AI systems and the scientific process to misinformation, raising concerns about trust and accuracy.