OAI-SearchBot Access
OAI-SearchBot is OpenAI's crawler that indexes the web for ChatGPT Search results. Per OpenAI's Publishers FAQ, OAI-SearchBot needs explicit access to your site to cite your pages in ChatGPT Search responses. SaaSalyst checks your robots.txt to verify OAI-SearchBot is not disallowed.
What SaaSalyst Checks
SaaSalyst fetches /robots.txt and parses it per RFC 9309. The scanner checks whether OAI-SearchBot is effectively disallowed — either through an explicit User-agent: OAI-SearchBot group with Disallow: /, or through a wildcard User-agent: * group with Disallow: / and no overriding allow for OAI-SearchBot.
Why This Matters
ChatGPT Search uses OAI-SearchBot to crawl public pages and surface them as answer citations. OpenAI separates this crawler from GPTBot (which trains models) — so blocking GPTBot for training data is reasonable; blocking OAI-SearchBot eliminates your product from ChatGPT Search entirely.
Many robots.txt files use Disallow: / under User-agent: * during staging or to block all crawlers, then forget to add explicit Allow groups when going live. The result: AI search systems cannot index the production site even though SEO crawlers can.
This is a high-severity check because the consequence is invisibility in a growing channel, and the fix is one extra block in robots.txt.
How to Fix It
- Add an explicit User-agent: OAI-SearchBot block in robots.txt with Allow: / or omit any Disallow: / directive in that group.
- If your wildcard rule is Disallow: /, ensure OAI-SearchBot's group does not inherit it: place Allow: / directly under the explicit group.
- Verify by fetching robots.txt manually and confirming the OAI-SearchBot block does not contain Disallow: /.
- After updating, monitor server logs for User-agent strings containing "OAI-SearchBot" — typical first crawl arrives within days.
Frequently Asked Questions
Should I allow OAI-SearchBot if I block GPTBot?
Often yes. GPTBot (training) and OAI-SearchBot (search) are different crawlers with different goals. Blocking GPTBot opts you out of model training data; blocking OAI-SearchBot opts you out of ChatGPT Search citations. SaaSalyst flags blocked OAI-SearchBot as high severity because the discovery cost outweighs the privacy benefit for most public-facing SaaS products.
How is OAI-SearchBot different from GPTBot?
GPTBot crawls pages to feed OpenAI's model training pipeline; OAI-SearchBot crawls pages to feed ChatGPT Search results — answer citations users see in real time. Per OpenAI's Publishers FAQ, the two have separate user-agent strings and can be controlled independently in robots.txt. SaaSalyst checks OAI-SearchBot specifically because it's the one that affects AI search visibility.
How does this check affect my Business Readiness Score?
SaaSalyst rates a blocked OAI-SearchBot as high severity in Distribution Readiness. AI search is a growing organic channel — getting cut out of it is comparable to having Googlebot blocked.
Check Your SaaS Now | Free
SaaSalyst scans your website in 30 seconds and checks for OAI-SearchBot Access along with 101+ other business readiness signals.
Scan Your App