Google's AI Overviews spew false answers per hour, bombshell study reveals
5 hours ago
- #Media Impact
- #AI Accuracy
- #Google Search
- Google's AI Overviews produce millions of inaccurate answers per hour, according to a study by startup Oumi.
- The analysis tested 4,326 results from Gemini 2 and Gemini 3 models, finding accuracy rates of 85% and 91%, respectively.
- With billions of searches expected, this translates to hundreds of thousands of errors per minute.
- Errors include basic factual mistakes, such as incorrect dates for events related to Bob Marley and Dick Drago.
- AI Overviews often cite unreliable sources like Facebook pages, blog posts, and Wikipedia entries.
- The feature is easily tricked into spreading fake news, as shown by an example involving a fabricated blog post.
- Citation issues worsened between models: 'ungrounded' answers rose from 37% in Gemini 2 to 51% in Gemini 3.
- Publishers criticize Google for using their content without compensation and for lacking fact-checking oversight.
- Google disputes the study, citing flaws in the benchmark test and questioning Oumi's methodology.
- AI Overviews have previously given bizarre advice, like adding glue to pizza sauce or promoting tobacco for kids.