From Five Parts to One Command

Building an AI-callable search for your own data on Cloudflare used to mean pulling together five things: Workers AI for models, AI Gateway for call management, Vectorize for vector storage, Cloudflare R2 for file storage, and Browser Run for crawling. You had to stitch them together yourself, and then handle the whole pipeline — crawling, chunking, embeddings, retrieval — on your own.

Two terms show up a lot below, so worth defining upfront: embedding turns text into a string of numbers (a vector), and search works by comparing these vectors to find meaningfully similar content, not just literal matches. reranking takes a first pass of rough results and runs another model over them to surface the most relevant ones first. Both run on every index build and every search.

Six New Capabilities

• Full-site indexing even without a sitemap: a sitemap is a file listing every page URL on a site — previously required for site-wide indexing. Now with Discover mode, it follows links across pages by itself, using Browser Run's /crawl under the hood.
• One public URL searches everything: turn on the public URL on a namespace (a container holding multiple search instances), and you get /search and /mcp endpoints that query every instance inside in one go. No auth required — safe to hand straight to a client.
• Custom domains: expose your endpoint as something like search.example.com/mcp.
• Want it private? Put Cloudflare Access in front: once enabled, queries require a login, so only authorized people or agents can get in.
• Hybrid search for better results: semantic and keyword search run in the same query. It handles open-ended questions ("what does this thing do?") and precise name lookups alike.
• EmDash sites: just install a plugin: EmDash is Cloudflare's open-source CMS. Add the AI Search plugin and your content gets semantic search out of the box.

Cloudflare Built an MCP from 10 of Its Own Sites

This section best illustrates what the service can actually do, because it fixes a problem that happens every single day.

AI coding assistants writing Cloudflare-related code draw on stale training data and routinely produce outdated API calls. The current workaround — making the agent hit a web search and read back entire pages — is, in Cloudflare's own words, slow, token-hungry, and prone to landing on the wrong or outdated sources.

Their approach, in three steps.

Step one: one instance per site

They created a dedicated search instance for each of ten of their own properties: developer documentation, the blog, API docs, the community forum, plus docs for Astro, Vite, Vitest, Hono, Replicate, and OpenNext (all under the Cloudflare umbrella). Creating an instance is one command; for sites without a sitemap, a flag makes it crawl links instead:

npx wrangler ai-search instance create cloudflare-community \
  --namespace dev-stack \
  --source https://community.cloudflare.com \
  --type web-crawler \
  --parse-type discover

Step two: flatten ten instances into one search

Two paths here. Write a Worker that binds the namespace and fans a single call out to all instances you name — that's the route they took, because they wanted to expose it as a tool inside an MCP server. Or write no code at all: flip on the public endpoint on the namespace, and you immediately get /search and /mcp covering every instance — no auth, no deployment.

Step three: attach a domain, lock it down if needed

The public endpoint ships with a default URL that you can swap for your own domain. And if that search shouldn't be public, put Cloudflare Access in front of it, and it becomes a login-gated private endpoint.

After those three steps, the result is a single address: drop it into your MCP config, and your coding agent can query current documentation across all ten sites in one call, with responses citing where each piece came from.

{
  "mcpServers": {
    "dev-stack": { "url": "https://stack.mcp.cloudflare.com/mcp" }
  }
}

Their own dogfooding goes further: the blog has long run on AI Search, and this round also brought the developer docs and cloudflare.com itself on board — all on hybrid search.

The Two Hardest-to-Estimate Costs Are Now Free

The design logic here is more interesting than the rate card itself.

Embedding and reranking run on every index build and every search — making them the hardest line items to predict. You can't know in advance how many searches users will run or how many results each one will re-rank. Cloudflare's move: when you use the default models or a model selected from the Workers AI catalog, these two steps cost nothing. The most unpredictable corner of the bill is simply gone.

Answer generation and query rewriting, meanwhile, are optional steps that run on a model you choose. They bill as Workers AI usage — or you can point any model through AI Gateway credits.

† One 5M-token pool per month, shared across all file types — images included. ‡ One 2k-query pool per month, shared between semantic and full-text queries.

Cloudflare's sample bill

On the Workers paid plan: a source with 20k documents (roughly 20M tokens), plus 1,000 images, at 30k semantic searches per month, using the default embedding and reranking models:

Read that number with its assumptions attached: Cloudflare estimates roughly 10 KB per doc and 1 MB per image, and chunking overlaps by ~10% during ingestion (hence the ×1.1 in the bill). Bigger docs or more images, and the number changes.

Three Limits to Know Before You Start

Site sources must be domains on your own Cloudflare account. To use a website as a data source, it has to be a zone under your account. Other ownership verification methods are planned, but there's no timeline. This one determines whether you can even use the product — don't discover it halfway through setup.

Crawling honors site crawler rules. Browser Run's /crawl does the work underneath, but it operates under its own distinct crawler identity, Cloudflare-AI-Search: it respects robots.txt, uses a fixed and public User-Agent, and abides by a site's existing crawler control policies.

The prices are preview; billing isn't live yet. The entire beta is free right now. Cloudflare states explicitly that it will email well in advance before paid billing begins, and that preview pricing may change before then.

One command is all it takes to spin up a trial:

npx wrangler ai-search create my-search \
  --namespace my-namespace \
  --source https://my-website.com \
  --type web-crawler \
  --hybrid-search

AI coding agents writing Cloudflare-related code draw on stale training data and keep producing outdated API calls. Cloudflare's fix: create a search instance for the dev docs, blog, API docs, community forum, and the docs for Astro, Vite, Vitest, Hono, and more — combine them in one namespace, and expose a single MCP endpoint.

embedding (turning text into vectors for comparison) and reranking (reordering results by relevance) run on every index build and every search — the hardest usage to predict. When you use Cloudflare's designated Workers AI models, both cost nothing. The most unpredictable part of the bill, gone.

That math rests on Cloudflare's assumptions — ~10 KB per doc, ~1 MB per image. Different usage, different numbers. And these are preview prices; they can shift before billing begins.