Over the weekend, Claude’s shared conversations suddenly appeared in Google search results.
On July 27th, I checked the rules for crawling Claude’s sharing pages and found a noindex directive that was hidden behind a block, preventing Google from accessing it.
The sharing method is blocked by robots.txt on claude.ai. These pages also submit an X-Robots-Tag header set to none, which Google’s robots meta tag guidelines treat as noindex and nofollow.
Because of this, the rules conflict. According to Google’s guidelines, the noindex tag only works if the crawler is allowed to access and read the page. If a page is blocked by robots.txt, it can still be indexed if other pages link to it, as Googlebot marks the URL without opening it.
Matching is not like AI companies. If Google finds a blocked URL elsewhere, it can list the URL without specifying noindex.
What’s going on
404 Media reported on Monday that the pages of Claude shared from the site: operators. TechCrunch reported that it had access to medical supplies, internal company documents, and files containing the names and phone numbers of elementary school children. Artifacts, documents and small applications built into Claude, show the second way.
Many references, including Decoder, have reported that the share pages do not have the noindex tag. As of July 27th, the noindex header is provided and sits after the robots.txt block. It is not clear if that article existed before the weekend, as I have not been able to confirm this.
That’s Crawl Rules Returns
Findings from my July 27 checks against claude.ai:
- The robots.txt file on claude.ai does not allow /share/* under the User Agent: * group, which does not have a separate Googlebot group, so the group that Google goes by.
- The live share URL responds with the header x-robots-tag: none to the GET request.
- A similar header appears when the request claims to be Googlebot, and the response includes Vary: User-Agent.
- The /public/artifacts/ path is not included in the list of robots.txt.
I did not verify that the artifact URLs return a page-level noindex, so the two methods are not comparable based on their controls.
Independent IT consultant Daniel J. Glover reported a similar conflict on July 26. My check shows the setup was still in place on July 27, after it was reported that the chats were no longer showing up in search results, but I couldn’t find out what was given to Google when those URLs were first found.
Why Blocking a Page Doesn’t Remove It
Robots.txt helps guide how search engines crawl your website, but does not determine what gets indexed. Google has been clarifying this for years.
For example, John Mueller explains that even pages blocked by robots.txt can appear in search results if other pages link to them. If you want to keep a page out of search results, using the “noindex” tag is the right choice. Martin Splitt also recommends avoiding putting both rules on the same page. Google recently emphasized this distinction again, as seen in their update last week.
Blocking URLs with robots.txt does not remove pages that are already indexed. To be removed permanently, Google’s guideline is to use the “noindex” tag that your search engine can read.
Answer from Anthropic
Anthropic told TechCrunch that shared links will only appear in search results if people post them in publicly accessible areas, and that links sent via private messages will not be identified.
Spokeswoman Amie Rotherham added that these links “are not predictable or available unless people choose to share them themselves.” This talks about one way that a public sharing URL can be found, but it doesn’t talk about the conflicting rules I found on July 27th.
Google spokesman Ned Adriance said search engines don’t decide which pages are made public, and that Google provides site owners with crawling and indexing controls and enforces those controls.
Why This Matters
Any conversations or artifacts you haven’t posted are still accessible to anyone with the link. For Claude, go to Settings > Privacy > Shared chats to see the complete list.
Not sharing a conversation disables a direct link, but removing a URL from Google is a different matter. As a Claude user, you cannot control what Googlebot does for you or submit a removal request, as this is controlled by Anthropic.
Artifacts are published in their own process, so unposting a discussion and unpublishing an artifact from that discussion are two separate actions.
Beyond settings changes, consider establishing rules around sharing AI conversations and artifacts. If the conversation contains something you wouldn’t post publicly, don’t share it with a link that may be listed later.
Looking Forward
The pattern is not unique to Anthropic. OpenAI removed ChatGPT shared chats from search in August 2025, and Google blocked Bard’s articles from indexing in 2023. Each case involved a publicly shared URL that went further than its owner might have expected.
Three companies have now had customer interviews when they were searched. This is reason enough to treat a publicly shared link as a public web page from the moment it exists.
Featured Image: Actors of Thousands/Shutterstock
