Google-Extended Access
AI assistants answer product questions with whatever sources their crawlers can read. SaaSalyst checks whether your robots.txt and page-level signals allow Google-Extended (Google's AI-training token) to fetch your site — the access layer that sits underneath any chance of being cited by Gemini.
What SaaSalyst Checks
SaaSalyst parses your robots.txt (RFC 9309 group matching) for rules that target the Google-Extended user-agent, and probes homepage reachability under that user-agent string. The check reports robots.txt-level and user-agent-level access — it does not claim to verify what the crawler's production infrastructure ultimately does. This check measures declared user-agent POLICY, not verified crawler behavior — for this token the attribution is looser (it can affect more than one product surface), so SaaSalyst weighs it at low severity.
Why This Matters
Blocking an AI crawler is a legitimate content-policy choice — but for a SaaS storefront it usually isn't a choice at all: it's an inherited template default or a copied robots.txt. When the block is unintentional, your product simply doesn't exist in that assistant's source pool while competitors do.
SaaSalyst surfaces the current policy so the decision is deliberate. A fail here means an explicit disallow or a hard user-agent-level wall; a warn flags ambiguous or partial rules.
How to Fix It
- Open your robots.txt and look for a 'User-agent: Google-Extended' group (or a wildcard group) with Disallow rules covering your marketing pages.
- If the block is unintentional, remove the disallow or add an explicit allow group for Google-Extended.
- If a CDN or bot-management layer (not robots.txt) is challenging the user agent, add an exception for it on your marketing pages.
- Re-scan to confirm the policy reads as allowed.
Frequently Asked Questions
Does allowing Google-Extended guarantee my site appears in Gemini?
No — and SaaSalyst deliberately does not claim it does. Access is a precondition, not a promise: the check verifies your declared robots.txt and user-agent-level policy, which is the layer you control. Whether the assistant cites you depends on its retrieval and ranking, which nobody outside the provider can verify.
Should every SaaS allow AI crawlers?
It is a policy decision. SaaSalyst's position is that the decision should be deliberate: most SaaS storefronts benefit from being readable by assistant crawlers, but paywalled or licensing-sensitive content may reasonably block them. The check exists so an inherited template default doesn't make the call for you.
Check Your SaaS Now | Free
SaaSalyst scans your website in 30 seconds and checks for Google-Extended Access along with 118+ other business readiness signals.
Scan Your App