On October 15, 2026, Cloudflare and OpenAI said a joint pilot running since July has added hundreds of millions of previously unknown pages to ChatGPT's search index. It matters to sites behind Cloudflare and to any business that wants to show up as a source in ChatGPT answers.
What we know
- The pilot: per Cloudflare's blog, it started in July 2026 to test whether signals from Cloudflare's network can help ChatGPT users discover new content. The post is co-signed by two Cloudflare and two OpenAI staff.
- The scale: a participating cohort of roughly 1% of Cloudflare's network took part. The post doesn't say how sites were selected.
- The result: according to Cloudflare, OpenAI discovered and indexed "hundreds of millions" of new pages. No other figures are given: not how many sites, nor how much traffic followed.
- The signals: filters that skip pages that don't exist, that redirect almost everything to the homepage, or that duplicate other pages; change signals that tell a crawler when a page is worth revisiting, still being tested; and flags for individual harmful pages, such as a fake login form planted by an attacker. Cloudflare says it flags that page, not the whole domain.
- Who ranks what: OpenAI independently evaluates and ranks the pages it discovers based on relevance and helpfulness, the post says.
- What isn't shared: per Cloudflare, the research uses metadata about the web, not page content for model training, and a signal about a page "doesn't grant permission to access it."
- What's next: better page selection, faster change signals, and measuring what happens after discovery. Cloudflare warns that finding new pages can increase crawling at first.
What changes and what doesn't
What changes is one way in. Beyond links and sitemaps, part of what ChatGPT search discovers can now come from what Cloudflare's network sees, at least for sites in the pilot. A useful page with few inbound links, like a product manual or a restaurant menu, has a better shot at being found.
What doesn't change is who controls access. OpenAI's documentation says sites that block OAI-SearchBot in robots.txt won't appear in ChatGPT search answers, though they can still show up as navigational links. There's also no sign-up button: the post doesn't explain how to join the pilot or whether it will expand. And being discovered doesn't guarantee being cited.
How to tell if it affects you
- Check your robots.txt. Open yourdomain.com/robots.txt and look for OAI-SearchBot. If it's blocked, or you block all bots, your pages won't appear in ChatGPT search answers. GPTBot is separate: per OpenAI, it crawls for model training, and you can block it without dropping out of search. OpenAI says robots.txt changes take about 24 hours to apply.
- If you're on Cloudflare, open AI Crawl Control and review which AI crawlers visit your site and which ones you allow or block. A rule set months ago may be keeping OAI-SearchBot out.
- Fix what these filters skip. URLs that don't exist but still load a page, catch-all redirects to the homepage, and duplicate pages are exactly the patterns the pilot learns to ignore. Returning a real 404 for missing pages and giving every URL its own content helps every crawler, Google's included.
- Look for compromised pages. If someone slipped a fake form into a forgotten section, that page can get flagged. Check the Security issues report in Search Console and search Google for `site:yourdomain.com` to spot pages you don't recognize.
- Watch your crawl logs. More OAI-SearchBot hits may be part of this discovery phase: Cloudflare says crawling can rise at first.
This checklist is this article's advice: neither Cloudflare nor OpenAI has published a guide to joining the pilot.
Related: Cloudflare Shipped 46 Launches: What Changes for Your Site
Sources
- Cloudflare, "Cloudflare and OpenAI: Making AI discovery better for everyone," October 15, 2026
- OpenAI, "Overview of OpenAI Crawlers"
Updates: if Cloudflare or OpenAI open the program to more sites or share more figures, we'll add it here with a link.