62% of AI Product Recommendations Disappear After One Consumer Question – New Clovion Data
Zahir Hasan didn’t have to tell me that his company’s numbers were wrong.
I had sent Hasan, COO of Oslo-based research firm Clovion AI, a how-to questionnaire about “Surviving the AI Funnel,” Clovion’s new study of how Claude, ChatGPT, and Gemini recommend the brand in conversation. The tenth question was the usual one, the kind of thing you ask all research teams. The report states that the three AI assistants disagree completely on product facts 15% of the time, based on 33 confirmed objections. Was 33 really enough to support the claim about which model usually sells the least product features and which one usually monitors?
Hasan’s answer was not to defend the number. It was a fix. “The real number is 330,” he replied. “The designer zeroed in on the structure.” The same floating decimal, he said, also converted 2,040 brands to “204” on page seven of the PDF I had sent before publication. An updated version is coming out this week. So, I got the math fixed first.
That’s a weird way to start a column about an AI research report, admitting before anything else that the draft report was wrong about it. But it is the most reliable method, because the correction means something that the statistics of the research topic will never know. Learning AI responds appropriately, whether you’re a marketer trying to find out if ChatGPT recommends your product or a researcher building a study on it, comes down to grasping the decimal point before building a strategy around it.
The Funnel, Recapped
Put the typo aside for a moment and the basic research stands. Clovion conducted a total of 69,120 interviews with three assistants in 36 B2B software and fintech categories, asking the opening question like “what are the best CRM tools?” and then one reality follows. Repeating the same question kept 90% of the recommended list unchanged. Adding the information of one typical consumer, something as clear as “in the small group,” only 28% were saved. Sixty-two percent of the products that made the first response went for the second.
I asked Hasan if a “small group” was chosen to produce that drop. It wasn’t like that. His team also tested “big business” and it was almost identical, about 72% either way, compared to about 10% when the question was simply repeated. The list is not stable. It is responsive, and especially in that the model has decided who the brand is really for.
That’s the part you have to live with if you’re doing SEO or brand strategy for a living. Being named by an AI response is not the same thing as being trusted by it. A model that puts you first on your CRM list can still cut it when the buyer is specific, and Clovion’s data says that happens more often than not.
Corrections Change the Position of a Smaller, Most Quoted Number
This is where the fixed decimal is so important to how you should read this study. The old figure, 33 confirmed collisions, was small enough that any claim each model was built on was standing on thin ice. Corrected, it is 330, and the breakdown of each model shared by Hasan is much clearer than the combined figure of 15% that the draft report leads: Claude lays down the characteristics of the product 160 times against more than 10 claims. ChatGPT takes 70 times slower and never overcharges. Gemini runs the other way, doing 80 miracles against less than 30 claims.
Hasan’s working theory, taken from a separate, unpublished study of Clovion in which each model gets its answers, is that Gemini is very dependent on marketing materials and video, and therefore tends to owe the brand anything it deceives. Claude and ChatGPT rely heavily on documentation and product pages, describing the core product accurately, and saying “I don’t have it” when a new feature is poorly written. If that holds up under the research Clovion hasn’t released yet, then the AI assistant’s error indicator about your product is a function of what kind of content you put in front of it, and where that content lives.
I’ve spent over 20 years telling clients that pitching well and pitching correctly are two separate issues. This is the clearest evidence I’ve seen that they are now the same problem, playing out within the same conversation, and that the fix depends on which assistant made the mistake.
Why Nobody Catches Zero Lost
Frederick Vallaeys has an article in his book “The AI-Amplified Marketer” that explains exactly why the discounted decimal survives until publication. An automated report once flagged “good performance” on a keyword because its cost per acquisition was performing much higher than the target. Somewhere in the process, high had changed for good, where high CPA is bad news, not good news. Anyone who recited the abbreviation would nod, because the sentence reads well even though its meaning has changed.
Vallaeys relates this to predictive processing research, the idea that fluent learners don’t finish each word, they predict what comes next based on context and move on. That’s how “teh” is read as “the” and the missing “not” passed by you. As Vallaeys puts it, our mental model of a sentence overrides the text in front of us. A confident, well-structured PDF is the easiest place in the world for that to happen, and a zero dropped in a layout file is a much smaller, more forgivable version of the same failure.
This is also why the correction is “less trusting of the report.” “Keeping a human pilot in the loop checking the number instead of the vibe of the surrounding role.” Thirty-three collisions and 330 collisions do not differ by more than a factor of ten. They support completely different levels of confidence about whether each model’s pattern is real. Two hundred and four species and 2,040 species are not the same study. If Clovion hadn’t caught it, and if I hadn’t asked, the smaller, shakier numbers would have been circulating as fact, quoted by exactly the kind of trading machine that should catch this.
What Clovion Doesn’t Want, and Why That’s a Good Part
The report is careful to state the link between how the model determines its validity and that it recommends “a strong, consistent association, not a proven causal law.” I pushed Hasan on what a true cause test would look like. His answer: change one thing, the public content of the product, leave everything else, and see if the behavior of the models corresponds to the products that no one has touched. Clovion hasn’t done that test yet. He also admitted that it is very unlikely, that the actual position of the product in the real world drives both how the model describes it and whether it is recommended, which would make the placement of the actual lump and the “perception” of the model a symptom, not a cause.
That’s an unusually clear answer from a company that sells AI visibility monitoring, and that’s exactly why I trust everything Hasan told me. He also didn’t have details on how fast AI’s idea of changing a product is after that product has changed its content. “We didn’t do a test before and after that,” he said. “Consider it worth the test, which is not guaranteed in X weeks.” Anyone who tells you they can promise a specific timeline for delivering a Claude or Gemini vision for your product is speculating, Clovion admits.
What You Should Actually Do About It
There are three things you should do, based on what Hasan told me and what the adjusted data supports.
First, track the entire conversation, not the first response. When you monitor AI visibility with a single command check, you measure the top of a funnel that loses 62% of its content one sentence later. Build your awareness on the following questions that your real buyers ask.
Second, prepare the assistants one at a time, one after the other. Hasan meant that a single content change would not eliminate all three models at the same time, because they draw from different sources. His suggested order: fix the ground truth errors first, as those are cheap wins, then go after the most important segment match combinations in your pipeline, testing each helper every few runs rather than hoping for any one answer.
Third, you can cite statistics you don’t follow from its source, including this one. Clovion’s report itself needs to be corrected on its technical number, which is very noticeable. Before creating a column, client desktop, or brief content on any percentage of AI research, ask where the basic figure comes from and whether anyone has checked the figure as it left the design software.
I’ve watched SEO go through a few of these times, from Panda to mobile-first indexing to slow zero-click search results. Each rewarded the staff who checked the main source instead of repeating the title number. The appearance of AI is formed in the same way. The brands that win the disappearing act written by Clovion won’t be the ones with the best press release about their AI Overviews strategy. It will be those who read the report closely enough to question what “33” really meant, and who continue to ask that question after it.
Zahir Hasan is the COO of Clovion AI, based in Oslo, Norway. A revised version of Clovion’s “Surviving the AI Funnel,” which shows the statistics in this column, is due this week.
