AI Answers About Your Locations Are Often Wrong – Check Before Customers Do
AI tools returned at least one false positive in about 64% of UK high street stores in tests carried out by the retailer, according to Searchable statistics shared with Retail Focus. The most common mistake has been to put businesses in the wrong postcode.
These tests point to something that many sales teams are not following. If AI systems like ChatGPT or Google’s AI Mode answer questions about one of your sites, there’s no way to know what they’re saying about your business unless you check it out yourself. They may confidently say the business is closed, list services they don’t offer, or give an incorrect address.
We’ve covered how to get visibility into AI search, including Dan Taylor’s work tracking AI data. This piece covers another part of the problem. It looks at what AI is already telling people about your properties and how they find them before the customer does.
From Standard to Definition
A traditional local search gives customers many options to compare, such as a map pack, reviews, your website, or competitors. Then, customers make their choice. Search AI combines this process into one integrated answer that fully describes the business before the customer visits the website or business profile.
Description is not the same as ranking; standards are subject to change, and descriptions may not always be accurate. Again, this seems to vary depending on the type of question.
Whitespark analysis found that AI Overviews appear in 15% of targeted local searches, as [personal injury lawyers in Phoenix]. For informational questions, it comes up 92% of the time, and for mixed questions, like hiring a lawyer after an accident, it comes up 97% of the time. In the same study, location packs showed 93% of direct location searches. Overall, AI Overview was more common in general and general informational queries than in specific geographic searches.
SOCi’s Local Visibility Index analyzed over 350,000 locations and found that ChatGPT recommends 1.2% of them. That’s a much lower rate than similar products appear in Google’s local 3-pack, by SOCi’s own rating. BrightLocal’s Local Consumer Review Survey found 45% of consumers now use ChatGPT or similar tools for local business recommendations, up from 6% last year.
What are the wrong answers?
Searchable released two sets of test results to trade media this month. In tests involving 165 London businesses, Searchable ChatGPT tested, Gemini, and Perplexity with 13,365 questions about services, contact information, size, and dates of establishment, and compared the answers to Companies House records and official profiles.
According to CMOTech, 93% of these businesses had at least one basic fact that was incorrect or missing. Half of small businesses have received at least one false positive, compared to 32% of large companies. The second survey surveyed UK high street retailers with more than 72,000 questions. Retail Focus reported that 1 in 16 answers were incorrect. Incorrect postcode errors occur at a rate of one in 10, even if the information is specified by city.
Search founder Chris Donnelly told Retail Focus:
“For a small brick-and-mortar retailer, if their online presence is mostly focused on their website and Google Business listings, that’s a narrow channel of information for AI programs to learn from and represent themselves in their responses.”
The error categories are similar to what business owners describe on Google’s support forums, reporting that they see incorrect information being presented with confidence, and some saying that the errors hurt their businesses.
The Blind Spot
Traditional search leaves behind a trail of data. Impressions and clicks are tracked in the search console; rates vary across trackers; and any decline often leads to an investigation. However, there is no report to alert businesses if the AI incorrectly states your location’s hours or says your business is closed.
Traditional search doesn’t know how to represent you, but it doesn’t have to. It directs customers to sources they can see and rate themselves, such as your listings, your site, and dated reviews. AI collapses those sources into one written response, and the customer reads it as fact. If it is incorrect, no ranking status can be viewed and no traffic sink will be tracked, because the error resides in the text that was never displayed.
Major consumer AI tools often don’t give businesses a local level warning when their information is incorrect. Google’s documentation states that AI answers may contain errors, and each AI Overview carries a link to the answer. This answer works for debugging but is not an effective monitoring system. It only works if someone notices the mistake first.
5 Systems, Not 1
The vetting process can be complicated across different AI search platforms because they don’t work the same way. AI Overview now appears as part of regular Google search results, AI Mode is a conversational experience right inside Google Search. Gemini works as Google’s unique AI assistant. Meanwhile, ChatGPT and Perplexity are separate products offered by OpenAI and Perplexity AI.
Each of these tools processes, selects, and aggregates information differently, so the same question can get different answers depending on which platform you use. This difference is also seen in Searchable sales data, Confusion was found to give wrong answers in 10% of cases, compared to 5% for Gemini and 4% for ChatGPT.
Examining one system tells you little about the others. The AI Overview checker area can still be poorly defined in ChatGPT.
How to Evaluate What Customers See
Start by thinking about questions your customers might have, such as hours, services, or whether the location is good. Write these questions down in a standard list so that each area is evaluated in the same way. Run questions using AI Overview and AI Mode on Google, as well as ChatGPT, Gemini, and Perplexity. Where possible, check without saved chat content or personalization, and follow the prompts and responses you receive.
Since the answers may vary each time, ask the same questions several times. Organize your findings into categories such as factual errors, missing information, or conceptual problems. Treat errors of fact and omission separately. Emotions and order of praise are matters of dignity, not easy fixes. Pay special attention to correcting errors that would prevent someone from visiting, such as incorrect hours or a branch reported as closed.
When citing sources, make sure their information is accurate. Correct the controlling information or request corrections when needed. Keep checking back often because updating the source today does not automatically update the AI responses. Use a consistent list of questions and a log to simplify the process, especially if you have multiple locations. Vary how often you check based on the number of locations and how often the information changes.
What to Do When the AI Goes Wrong
Finding the fault is the first step. Fixing it is about carefully adjusting the inputs of these dependent programs, and testing whether the responses improve. Remember, each adjustment brings you closer to better, more reliable results.
If the answer is doing your wrong hours, make sure your Google Business Profile, your website, and your location pages are consistent and realistic. The conflicting details in all of these is one of the obvious reasons for the answer to be wrong. Same name, address, and phone Number consistency is what local SEO has always needed as a foundation, and it gets more complicated when you use multiple locations.
After editing the sources you control, look at the sources you don’t own. If the answer cites your business, open it and check its date and what it actually says. It could be extracting information from an old directory page, a review listing, or an about page for a similarly named business. Fixing these problems will likely involve direct communication.
This is where the size gap that Donnelly points to starts to make sense. Small businesses made more mistakes in Searchable’s testing, and his learning was that the small footprint of third-party information gives these programs little to work with. Our presentation in relation to ChatGPT quotes reaches the same conclusion. A full, consistent presence in all areas studied by these systems gives them better things to describe.
If you find accuracy is not a problem, and you are too concerned about being praised at all, that is the side of visibility. Dan Taylor wrote for Search Engine Journal on tracking how AI represents your brand over time and how discovery changes as searches are personalized for each user. These are the places to start in the question of “how do we win here”.
Looking Forward
Monitoring tools for AI responses are still in their early stages. Most can tell you what you said and how often, as opposed to telling you whether your speech is correct.
Finding out what the answer really says about the place will still come down to reading for yourself. As these tools improve, that may change, but for now, research is done manually, and if you find something wrong, the first move is to improve the sources you control.
Additional resources:
Featured Image: Natalya Kosarevich/Shutterstock
