In 2024, Google began prioritizing artificial intelligence-generated responses at the forefront of its search results page. This innovative feature, named AI Overviews, marked a significant shift for Google, evolving its role from merely an information aggregator to that of a content publisher.
A recent study evaluating AI Overviews indicated that these responses were correct nearly 90% of the time. However, considering that Google handles over 5 trillion searches annually, this translates to millions of incorrect answers being generated every hour, as reported by an AI startup called Oumi.
Notably, over half of the accurate responses flagged by Oumi were deemed “ungrounded,” meaning they were linked to sources that did not adequately substantiate the provided information. This raises concerns regarding the verification of the accuracy of AI Overviews.
While some technology experts assert that Google’s AI Overviews have shown reasonable accuracy and improvements in recent months, there are concerns that the average user may not be aware that these results require further verification.
At the request of The New York Times, Oumi assessed the reliability of Google’s AI Overviews utilizing a benchmark test known as SimpleQA, which is widely recognized in the industry for evaluating AI system accuracy. The analysis was conducted in October using the AI technology Gemini 2 for complex queries, followed by another evaluation in February after the system was upgraded to Gemini 3, a more advanced AI framework.
In both assessments, Oumi examined 4,326 Google searches and found that the accuracy rate was 85% with Gemini 2 and improved to 91% with Gemini 3.
Google has acknowledged the possibility of errors within its AI Overviews. However, the company has criticized Oumi’s findings, claiming that the analysis was flawed due to its reliance on a benchmark test developed by OpenAI, which contained inaccuracies itself. “This study has serious holes,” remarked Ned Adriance, a spokesperson for Google.
AI Overviews deliver two types of information: direct answers to inquiries and references to websites that corroborate those answers. During the analysis, Oumi identified that Facebook and Reddit were the second and fourth most frequently cited sources, respectively. When Google’s AI Overviews were correct, they referenced Facebook 5% of the time, while incorrect answers linked to Facebook 7% of the time.
To assess the accuracy of AI systems, organizations like Oumi utilize their own AI technologies to validate each response. Nonetheless, this approach has its limitations, as the AI system performing the verification may also produce errors.
