Claude Search Optimisation
Last updated View as Markdown
Claude can search the web while it answers and cites the pages it uses inline, and the evidence points to Brave Search, not Google or Bing, as the index behind it. Anthropic has not confirmed this, but it lists Brave Search among its subprocessors, and independent tests have matched Claude’s citations to Brave’s results.1 For content strategy that is the central fact: Google rankings do not translate directly into Claude citations. Web search launched in the United States in March 2025 and went global across all plans in May 2025.2
Does Claude really run on Brave Search?
The evidence is strong but Anthropic has not confirmed it, and the distinction is worth keeping.
In March 2025 Anthropic added Brave Search to its published subprocessor list, the register of partners that process Claude data.1 Independently, the programmer Simon Willison found that Claude’s citations for a test query matched Brave’s results for that query exactly, and identified a BraveSearchParams parameter inside Claude’s web search function.1 Anthropic itself documents three crawlers and describes Claude-SearchBot’s role, but has never publicly named its search provider.3
So: a legal disclosure plus independent technical replication, and no official confirmation. That is more than enough to plan content around, and it is not the same as Anthropic saying so. Treat the arrangement as evidenced rather than announced, and as something that could change without a press release.
The list has since grown a second entry under the same heading. As of 2 September 2026 Anthropic names two subprocessors under Web Search: Brave Search, scoped to all products, and TurboPuffer, scoped to all products except Claude for Government.4 Anthropic does not say what either one contributes, so this complicates the picture rather than reversing it. Brave is still listed, still unrestricted, and still the only one independently matched to Claude’s citations; it is no longer the only named component of Claude’s web search.
You may also encounter a widely repeated statistic putting the overlap between Claude’s citations and Brave’s top results at 86.7%, with a p-value attached. It traces back to a single Profound blog post published on 21 March 2025, the day after Claude’s web search launched, which reports “86.7% (13 out of 15 total results)” across three commercial “best X” queries.5 The direction it points is right; the precision is not real, and we do not rely on it here.
How does Claude retrieve and cite?
When a query requires current information, Claude reformulates the user’s question into a more search-optimised form and issues it to the search backend rather than passing the original prompt through. It then retrieves candidate results and, where necessary, fetches specific pages to read in detail. Two consequences follow: a page must be present in the index Claude queries, and it must hold up when read. Ranking gets you considered; the content earns the citation.
Search is not the only route by which Claude reaches a page. Claude in Chrome, an extension that reads and acts on the page a user is viewing, became generally available on 18 December 2025 for Pro, Team and Enterprise plans.6 That path is browsing rather than retrieval, so it does not bear on the citation mechanics below, but it does mean Claude can read a page that never entered Brave’s index.
Citations appear inline within Claude’s prose as numbered references, embedded where the content is relevant to the point being made. This differs from Perplexity’s sidebar source list and ChatGPT’s citation list appended to the end of the response. Inline placement means citations appear where users are actively reading, giving cited sources contextual visibility rather than a footnote position.
What are Anthropic’s three crawlers?
Anthropic documents three crawlers, each with a distinct purpose.3 Conflating them is a common mistake with direct consequences for robots.txt decisions.
ClaudeBot collects web content that may be used to train Anthropic’s models. Blocking it excludes your site from future training data. It has no effect on whether Claude cites you in a search response.
Claude-SearchBot “navigates the web to improve search result quality for users”.3 This is the crawler that matters for citation visibility. Blocking it reduces your chance of appearing in Claude’s search responses.
Claude-User fetches a page live, at the moment someone asks Claude a question that requires it, such as pasting a URL and asking for a summary. It sits outside the search indexing pipeline, but Anthropic states that disabling it “prevents our system from retrieving your content in response to a user query, which may reduce your site’s visibility for user-directed web search”.3
Two of the three therefore bear on visibility. Claude-SearchBot is the indexing crawler, and Claude-User needs to stay allowed alongside it for live fetches to reach your pages. Blocking ClaudeBot, the training crawler, has no effect on either.
There is no documented fourth crawler
A number of AI bot directories confidently list a fourth Anthropic crawler, claude-code, for the Claude Code CLI, and describe it as officially documented. Anthropic's own crawler documentation names three agents and does not mention it. Some of those directories also claim that blocking claude-code "prevents AI training", which makes no sense for an on-demand developer tool. Treat the bot-directory sites with caution: several are themselves AI-generated. Manage the three agents Anthropic actually documents.
What does Claude favour?
Brave Search ranking is the documented part. Before any of Claude’s citation logic applies, your page must be in Brave’s index and rank for the query Claude generates internally. Brave serves its web results from its own index: it removed the last of its Bing API calls in April 2023, which had accounted for roughly 7% of results.7 Brave publishes no ranking documentation, so what it weights is not known beyond the fact that its crawler follows Googlebot’s access rules.8
Everything after retrieval is undocumented. Anthropic has published how Claude’s web search works at the level of crawlers and plans,23 and nothing about which retrieved pages it cites or why. The claims in circulation, that Claude weights authorship more than other surfaces, is “conservative with citations”, or prefers a neutral register, come from tool and agency blogs that name no method, and several of them repeat this page’s own earlier wording. Treat them as unmeasured.
What that leaves is the general evidence, which applies here as anywhere: the original GEO paper found that adding statistics, quotations and citations to a page lifted its share of a generative answer it was already retrieved for; a 2025 retrieval benchmark found embedding models lose 15.6% of retrieval performance when the relevant information sits late in a passage; and C-SEO Bench, a simulated-engine test of the same edits, found they did not move where a source was cited. Generative engine optimisation carries those sources and their scope. The practices that follow, a named author a reader can check, claims that link to their primaries, sections that open with the answer, and a register that informs rather than sells, are good for the reader whatever Claude does with them, and nothing measured on Claude contradicts them.
How do you optimise for Claude citations?
-
Verify Brave indexing. Search for your key pages on Brave directly. If they do not appear for relevant queries, Claude is unlikely to surface them regardless of other signals. Brave takes submissions and re-fetch requests at search.brave.com/submit-url. Two details in Brave’s crawler documentation matter more than the form: its crawler “does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them”, so there is no Brave-specific robots.txt rule to write, and “if a domain or page is not crawlable by Googlebot, then Brave Search’s bot will not crawl it either”.8
-
Allow Claude-SearchBot and Claude-User in robots.txt. If you have historically blocked all AI crawlers or applied wildcard rules, verify that neither is inadvertently blocked. Each Anthropic user agent must be managed independently, and the directives have to be repeated for every subdomain you want covered.9
-
Establish author entities in schema. Add
authormarkup to Article schema with sufficient structured data for Claude to resolve the author:name, aurlpointing to a personal site or profile, andsameAslinks to LinkedIn or institutional pages where relevant. -
Write for passage extraction. Open each H2 and H3 section with one or two sentences that directly answer the question the heading raises. Long preambles before the substantive answer reduce the chance of that passage being selected.
-
Cite primary sources outbound. Link to original research, official documentation, and named experts. Claude’s citation behaviour favours sources that themselves demonstrate editorial rigour.
-
Build authority through traditional means. Editorial backlinks, brand recognition, and original content all improve Brave rankings, which is the prerequisite for Claude visibility. The same off-page work that improves Google rankings improves Brave rankings, though relative weighting differs between the two indexes.
How do you measure Claude visibility?
There is no Claude equivalent of Google Search Console. Referrer tracking gives a direct signal: sessions from Claude appear in analytics with claude.ai as the referrer. As of May 2026, Google Analytics 4 aggregates traffic from major AI tools including Claude into a dedicated ‘AI Assistant’ channel, making this visible without custom configuration.10
For proactive measurement, sample manually. Run representative queries for your niche in Claude and record which sources are cited. For brand-sensitive queries, tracking this periodically gives a directional indicator of whether Claude cites your content for those queries.
Frequently asked questions
Does blocking ClaudeBot affect Claude citations?
No. ClaudeBot is the training crawler only. The two agents that bear on citation are Claude-SearchBot, which indexes, and Claude-User, which fetches live in response to a question.
If my site ranks well on Google, will Claude cite it?
Not automatically. Google and Brave rankings correlate imperfectly. A page can rank on the first page of Google and not appear in Brave for the same query. Checking Brave directly is the only reliable way to confirm indexing status.
Is Claude a significant traffic source?
Referral volumes from claude.ai are currently smaller than from Perplexity or ChatGPT Search for most sites. The more immediate value is visibility with technical and professional audiences, who use Claude at above-average rates relative to the general search population.
Does Claude use Google’s index at all?
Not by way of Brave’s web results, which come from Brave’s own index; Brave removed its remaining Bing API calls in April 2023.7 One nuance often stated too strongly: Brave does still offer Google Fallback mixing in its consumer search product, but it is an opt-in setting a user can enable, not a default, and it is a property of the consumer product rather than of the index Claude queries.7 For content strategy the conclusion is unchanged: get indexed by Brave.
Footnotes
-
Anthropic appears to be using Brave to power web search for its Claude chatbot — TechCrunch ↩ ↩2 ↩3
-
Claude web search now available globally on all plans — Anthropic ↩ ↩2
-
Does Anthropic crawl data from the web, and how can site owners block the crawler? — Anthropic Help Center ↩ ↩2 ↩3 ↩4 ↩5
-
Subprocessors — Anthropic Trust Center. As of 2 September 2026 the list of 20 carries Brave Search and TurboPuffer as its two Web Search entries; Anthropic publishes no addition dates on the list itself. ↩
-
Claude web search explained — Josh Blyskal, Profound (21 March 2025) ↩
-
Claude for Chrome — Anthropic. Generally available to Pro, Team and Enterprise plans from 18 December 2025. ↩
-
Brave Search removes last remnant of Bing from search results page — Brave ↩ ↩2 ↩3
-
Anthropic clarifies how Claude bots crawl sites and how to block them — Search Engine Land ↩