Study finds ChatGPT and Google AI Overviews cite different sources for the same tools

DerivateX analyzed 402 AI citations from 15 buyer queries and found that ChatGPT and Google AI Overviews often recommend the same software, but rely on almost entirely different evidence. The findings suggest brands need separate visibility strategies for each AI engine.

Why it matters: - Software buyers are using AI search to compare tools, and a win in one engine does not reliably carry over to the other. - The study suggests ChatGPT and Google AI Overviews function more like separate discovery channels than one shared search surface. - Brands that track AI visibility on only one engine may overestimate their real reach.

What happened: - DerivateX analyzed 402 individual citations across 15 buyer-intent queries in five segments of the cloud media and content infrastructure category. - The study, titled The Agreement Gap, compared ChatGPT and Google AI Overviews across open-ended buying questions. - Across those queries, the two engines named the same tools about 32% of the time. - The two engines cited the same web pages only about 4% of the time. - DerivateX published the full dataset alongside the report for journalists and researchers.

The details: - Researchers submitted each query to both engines between five and ten times to reduce answer variability. - On open-ended discovery questions, brand overlap averaged 38%. - On problem-solving questions, brand overlap fell to 22%. - On direct comparison queries naming three tools, source overlap rose only to about 30%. - On open-ended queries, source overlap dropped to around 4%. - ChatGPT leaned heavily on community discussion, with Reddit and forums making up about 25% of its citations. - ChatGPT referenced Reddit 39 times across the query set. - Google AI Overviews drew only about 4% of its citations from community sources. - Google AI Overviews referenced Reddit 7 times. - Google AI Overviews favored vendor and competitor pages, which accounted for roughly 45% of its citations. - Google AI Overviews cited about 16 sources per answer on average. - ChatGPT cited roughly 10 sources per answer on average. - Google AI Overviews pointed to a tool's own website in about 20% of citations. - ChatGPT did so in about 15% of citations. - When researchers pooled everything ChatGPT cited, including primary and supporting sources, the combined set still matched only about 25% of the sources Google AI Overviews placed front and center. - Roughly 75% of Google's primary sources did not appear anywhere in ChatGPT's answers. - Vimeo, Wistia, Gumlet and Cloudinary were the most consistently named tools across both engines. - On problem-solving questions, both engines often answered without naming any product. - The company also published a list of the best B2B SaaS SEO agencies for teams adapting to this two-engine environment. - The full benchmark, including methodology, charts and downloadable dataset, is available on the DerivateX website.

Between the lines: - The results point to a split in how the engines decide what counts as credible support. - ChatGPT appears to draw more from community sentiment and discussion. - Google AI Overviews appears to lean more on structured vendor and comparison content that already ranks well in search. - That means source strategy may matter as much as product mentions for companies trying to be discovered through AI search. - Apoorv Sharma, co-founder of DerivateX, said brands assume a win in one AI engine carries over to the rest, but the data shows the opposite.

What's next: - DerivateX says companies should treat ChatGPT and Google AI Overviews as separate optimization targets. - The firm says visibility work for ChatGPT should emphasize community presence and sentiment. - The firm says visibility work for Google AI Overviews should emphasize structured, indexable vendor and comparison content. - The company expects the published dataset to support further research and verification.

The bottom line: - ChatGPT and Google AI Overviews may name many of the same products, but they do not use the same evidence to get there.