NLWeb (Natural Language Web)

NLWeb lets AI agents ask a website questions in plain language and get structured answers back, built from the Schema.org and RSS data the site already publishes. It suits sites with rich structured data, such as commerce, media and documentation sites; purely editorial sites do not need it. Where WebMCP exposes callable tools in the browser at the moment a user’s agent takes action, NLWeb is a query layer on the server. It was released as open source by Microsoft in May 2025, developed by R.V. Guha, who previously created RSS, RDF, and Schema.org, and has since been integrated into Cloudflare’s AI Search product.¹²

What does NLWeb do?

NLWeb takes the structured data a site already publishes (Schema.org JSON-LD, RSS, other JSONL formats) and makes it queryable by AI agents through two endpoints:

• /ask: a REST endpoint that accepts natural language queries and returns structured answers in Schema.org format
• /mcp: an MCP server endpoint that makes the site discoverable and callable by any MCP-compatible agent

Microsoft says every NLWeb instance “is also a Model Context Protocol (MCP) server”,¹ so any MCP-compatible agent can connect to a site running it without crawling individual pages.

The processing pipeline, as the project documents it: a natural language query arrives at /ask or /mcp, the server queries a vector database for candidate items, an LLM scores them against the query, and the results are returned as Schema.org JSON.³ The agent never reads individual HTML pages.

What this requires in practice: a vector database (Qdrant, Snowflake, Milvus, Azure AI Search and Elasticsearch are among those supported), LLM connectivity (the protocol works with most major models including Deepseek, Gemini, and Claude), and the NLWeb server itself (open source, available on GitHub).⁴ Cloudflare’s AI Search integration removes the vector database requirement for sites already on Cloudflare.²

How does NLWeb differ from WebMCP?

The two protocols address the same question (how can AI agents interact with a website?) from different positions.

Both produce an MCP server endpoint; neither replaces the other. A site running NLWeb exposes a queryable catalogue. A site implementing WebMCP exposes callable actions (add to cart, check availability, book a service). The two are complementary layers in the same agentic stack.

What does an NLWeb query look like?

A user’s AI agent sends a POST request to a travel site’s /ask endpoint:

{ "query": "beach resorts in Thailand with kids activities under £200" }

The server’s vector database retrieves relevant LodgingBusiness schema.org entities. The LLM ranks and filters them. The endpoint returns a Schema.org-formatted JSON response the agent can present directly. The agent did not crawl individual hotel pages; it queried a structured catalogue.

The same pattern applies to recipe sites (query: “pasta dishes under 30 minutes”), media archives (query: “long reads about climate policy published in 2024”), and documentation (query: “how to configure webhook retries”).

Which sites should implement NLWeb?

Value scales with the depth of the content catalogue and the degree to which users (or agents) need to browse or filter it rather than retrieve individual pages.

Good candidates:

• Commerce sites. Product catalogue queries are the primary use case: an agent asking for “waterproof running shoes under £120 in size 9” against a Product catalogue, rather than filtering a faceted navigation. Tripadvisor and Shopify are named early adopters.⁵
• Media and publisher sites with large archives. Deep catalogues of recipes, reviews, or long-form content benefit from queryable interfaces. Named early adopters include DDM (Allrecipes, Serious Eats), O’Reilly Media, and Hearst (Delish).⁵
• Documentation and knowledge bases. AI coding tools benefit from queryable structured responses when working with API references and guides, the use case Google’s John Mueller gave for Google’s own Markdown pages on developers.google.com: coding tools “can be (I think) efficient and accurate with the code they produce if they can easily read / parse reference material”.⁶
• Events and ticketing. Eventbrite is a named adopter; date and location queries against a live events catalogue are a natural fit.⁵

Not a good fit:

• General editorial and informational sites. If the site’s primary value is individual article retrieval rather than catalogue browsing, agentic SEO content optimisation is the more relevant investment. NLWeb infrastructure does not improve how individual articles are found or cited in AI-generated answers.
• Sites with thin or unreliable schema.org markup. NLWeb’s responses are only as good as the structured data underneath. Poor schema produces inaccurate or unhelpful NLWeb responses. Audit and strengthen schema first.

Does NLWeb affect search rankings?

No. NLWeb exposes a queryable API for agents to call, not something a search crawler reads. Google’s AI optimisation guide, which lists the agentic protocols it does track, does not mention it,⁷ and Microsoft’s announcement presents it as a way for people and agents to query a site, not as an input to Bing or Copilot.¹ The value is direct agent access to site content, not improved search visibility. Do not implement NLWeb expecting SEO or AI citation benefits.

How effective is NLWeb vs. standard HTML crawling?

An academic comparison first posted to arXiv in November 2025 and since published at the ACM Web Conference 2026 tested MCP-based retrieval, NLWeb, RAG, and raw HTML against the same query set.⁸ Its finding is that the interface matters and the choice between the three non-HTML interfaces does not: “F1 rises from 0.67 for HTML to between 0.75 and 0.77 for the other agents”, a band covering RAG, MCP and NLWeb alike, while token usage falls from about 241,000 for HTML to between 47,000 and 140,000 per task. So the study supports NLWeb over raw HTML crawling; it does not show NLWeb outperforming a well-built RAG pipeline. The caveat: this is a controlled benchmark, not a production deployment study. Production results vary with schema quality and query type.

Current state

NLWeb was announced at Microsoft Build in May 2025 and has been in active development since.¹ As of May 2026:

• GitHub: 6,200+ stars, actively maintained, with R.V. Guha among the direct contributors⁴
• Cloudflare integration: Cloudflare’s AI Search product supports NLWeb deployment, removing the need for separate vector database infrastructure for sites already on Cloudflare²
• Named early adopters: Tripadvisor, Shopify, Eventbrite, O’Reilly Media, DDM (Allrecipes/Serious Eats), Hearst (Delish), Chicago Public Media, Common Sense Media⁵
• Protocol compatibility: works with most major LLMs and vector databases; every NLWeb instance exposes an MCP endpoint

The protocol is open-source and has no single controlling vendor beyond Microsoft’s initial development. The reference implementation is now hosted under an nlweb-ai GitHub organisation rather than microsoft, with Microsoft still listed as the project’s support contact and trademark holder.⁴

What to do now

Commerce and media sites: assess whether your Schema.org coverage is solid enough to serve as the data source. A schema audit should come before any NLWeb implementation; the quality of your structured data determines the quality of NLWeb responses. Sites already on Cloudflare can review the Cloudflare AI Search + NLWeb documentation as the lowest-friction path to deployment.

Documentation sites: NLWeb is worth evaluating if your audience includes AI coding tools that read documentation to generate code. The structured query interface fits how these tools consume reference material.

Editorial and informational sites: no implementation needed. Monitor as platform-level integrations mature (Cloudflare’s integration is already a sign that NLWeb may become a hosting-layer default rather than a DIY implementation). The relevant investment remains content quality and agentic SEO optimisation.

Frequently asked questions

Does NLWeb require rebuilding the site?

No. NLWeb reads existing structured data: Schema.org JSON-LD, RSS, JSONL. The quality of your schema determines the quality of NLWeb responses. The implementation work is infrastructure (vector database, LLM connectivity, NLWeb server), not content restructuring.

Is NLWeb the same as MCP?

Related but distinct. Every NLWeb instance is an MCP server: it exposes an /mcp endpoint that MCP-compatible agents can connect to. NLWeb adds the /ask REST interface and the schema.org retrieval pipeline on top. An MCP server is not necessarily an NLWeb instance; an NLWeb instance is always also an MCP server.

Does Google support NLWeb?

Microsoft developed and maintains NLWeb; Google has not announced adoption. NLWeb and WebMCP are complementary (both eventually produce MCP endpoints) but have separate governance and different implementation paths.

Is NLWeb a standard or a product?

It is an open-source protocol. Microsoft published the code and the specification on GitHub under an open-source licence. Any organisation can implement NLWeb without licensing fees or vendor lock-in. The protocol’s design builds on established web standards (Schema.org, RSS, MCP) rather than proprietary formats.

How does NLWeb relate to llms.txt?

They address different layers. llms.txt is a discovery hint: a Markdown index of what a site contains. NLWeb is an action layer: a queryable API that returns structured answers. A site could publish both; they do not conflict. See llms.txt for the index convention, and note that Google’s AI optimisation guide says such files will “neither harm nor help your site’s visibility or rankings in Google Search”.⁷

Footnotes

1. Introducing NLWeb: bringing conversational interfaces directly to the web — Microsoft News ↩ ↩² ↩³ ↩⁴

2. Get started with NLWeb — Cloudflare AI Search documentation ↩ ↩² ↩³

3. Life of a chat query — NLWeb documentation, GitHub and NLWeb REST API, which names the two endpoints. ↩

4. NLWeb — GitHub ↩ ↩² ↩³

5. NLWeb pioneers: success stories and use cases — Microsoft Tech Community ↩ ↩² ↩³ ↩⁴

6. Mueller explains why Google uses Markdown on dev docs — Search Engine Journal ↩

7. Optimizing your website for generative AI features on Google Search — Google Search Central ↩ ↩²

8. MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web — arXiv 2511.23281. Submitted 28 November 2025; published in the Proceedings of the ACM Web Conference 2026. ↩