Testing suggests Google's AI Overviews tell millions lies per hour
6 hours ago
- #Gemini AI
- #AI Accuracy
- #Google Search
- AI Overviews is a Gemini-powered search feature introduced in 2024, facing user criticism over accuracy issues but is improving.
- A New York Times analysis using the SimpleQA evaluation found AI Overviews is correct 90% of the time, but 1 in 10 answers is wrong.
- The benchmark was conducted with startup Oumi, showing accuracy rose from 85% with Gemini 2.5 to 91% with Gemini 3.
- Despite improvements, the error rate translates to tens of millions of incorrect answers daily when scaled to all Google searches.
- Examples of errors include providing wrong dates for Bob Marley's former home museum and misrepresenting Yo Yo Ma's hall of fame induction.