Company apologizes after AI support agent invents policy that causes user uproar
a day ago
- #AI Hallucinations
- #Tech Backlash
- #Customer Support
- A developer noticed Cursor, an AI-powered code editor, logging them out when switching between devices.
- Cursor support, represented by a bot named 'Sam', falsely claimed this was a new security policy.
- The AI's made-up policy led to user complaints and subscription cancellations on Hacker News and Reddit.
- This incident highlights the issue of AI 'confabulations' or 'hallucinations' where AI invents false information.
- AI models often prioritize confident, plausible responses over admitting uncertainty, leading to misinformation.
- The lack of human oversight in customer-facing AI systems can result in immediate business damage.
- Users expressed frustration over the non-existent policy, with some canceling subscriptions and workplaces removing Cursor.
- The Reddit thread discussing the issue was eventually locked and the original post removed by moderators.