AI keeps inventing fake cases. Lawyers keep citing them
2 hours ago
- #Human Trust in AI
- #Legal Sanctions
- #AI Hallucinations
- Alabama Supreme Court sanctioned a lawyer for repeatedly using AI-generated legal briefs with false citations, including nonexistent cases.
- More than 1,400 court cases in three years involved AI errors, with filings by attorneys and self-represented litigants, showing a recent plateau but ongoing judicial rulings.
- AI errors have impacted journalists, software developers, researchers, and government consultants, such as a book author admitting to fabricated quotes.
- People often trust AI even when aware of its fallibility, leading to legal dismissals, fines, job losses, and system outages, with experts warning of higher stakes.
- Humans tend to overtrust AI, viewing it as infallible, and AI-generated false but realistic answers can be unusually persuasive, as shown in studies.
- Research indicates this trust extends to critical scenarios like drone warfare simulations, where participants reversed accurate judgments based on random AI feedback.
- AI's versatility allows users to defer thinking entirely—a 'cognitive surrender'—and feedback or rewards only partially reduce reliance on faulty AI.
- Education about AI limitations has limited impact; warnings in studies show mixed results, with factors like trust and time constraints overriding awareness.
- Workplace pressures and AI advertising promote its use for efficiency, potentially reducing verification and reinforcing unwarranted confidence as AI improves.