The additional outbound clicks that websites receive when Google’s AI Overview is removed appear to be of the same quality as the clicks that are still aggregated with AIOs. This finding comes from an updated, randomized field test, which showed a 39.8% decrease in organic clicks when summaries were displayed.
Researchers Saharsh Agarwal and Ananya Sen have updated their working paper to include new analyzes of click quality, treatment changes, and question types. The newspaper reported that it was down 38% when SEJ first covered it in April.
Updated statistics show click-throughs per search increasing and zero-click searches increasing when AI Overview is displayed. Because AI Overview is triggered on about 41% of queries, that amounts to a measurable reduction in outbound clicks for all searches.
What Click Quality Data Shows
By clicking on the questions where the AI Overview would appear, the authors compared how often people hit the back button to return to the results page, how often the visit ended within 10 seconds without interaction, and the total time spent on the local site.
None of the three showed statistically significant differences between groups. About 4 out of 10 clicks of the same tab led back to the results page in both cases, about 18% of visits ended within 10 seconds, and time on site was statistically indistinguishable.
Google VP of SearchLiz Reid, said AI Overview reduces “bounce clicks” but has yet to release supporting data Claim. The test finds the same question on the other side. If the snapshots took more low-value visits, more clicks in the no-Upview group must look worse, but it doesn’t.
The authors write that the result “contradicts the idea that AIOs primarily eliminate visits to low-engagement websites.”
Disclosure: the revised paper cites two Search Engine Journal articles in its discussion of Google’s claims.
Behavior Reversed When Groups Change
After the first two-week period, the researchers rotated the assignments around.
Participants who were given the AI Overview saw their clicks per external search decrease, while those who did not receive it saw their clicks per external search increase. Interestingly, the zero click rates were the same.
Knowledge Questions Carry the Result
When the authors examined the data by query type, they found that the loss focused on information searches, while removing AI Overview increased clicks per search. Navigational and functional questions showed no measurable change, although those samples were small.
An overview of AI was recorded or intended to be implemented in 53% of information queries, compared to 15% of navigation and 6% of tasks. The ranking breakdown adds that the top three results received the most clicks when the Top Page Overview was removed, the first position almost doubled.
Why This Matters
Google’s answers to questions about lost traffic depend on increasing the quality of clicks. This experiment tested the claim and found no measurable difference.
Looking Forward
The authors wrote that the aggregate traffic loss can increase if the AI Overview appears in many queries over time. The paper is a draft submission to SSRN and has not yet completed peer review.
Featured Image: takasu/Shutterstock
