Is your site agent-ready? →

Home › Tools › The Agent-Reachability Check

Free check · 6 items · 4 minutes

Can AI fetchers reach and read the pages you rank with?

Reading your robots.txt tells you what you published. It does not tell you what your edge did. Six items, four numbers, and your matrix against a live page one we measured.

6 items 4 optional numbers Measured against 35 live fetches No email

No email required. Nothing is sent anywhere — your answers stay in this browser.

0 of 6 answered

The Agent-Reachability Check

Answer from a measurement you have taken, not from a framework's reputation. “Not sure” scores as no.

Item 1

Can you produce, today, a matrix of HTTP status codes returned to GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and CCBot across your top 20 templates?

Direct HTTPS fetches, one per user-agent per template. It costs an afternoon and no API units.

Item 2

Have you compared that matrix against your robots.txt, and confirmed the two agree?

Token by token. Where they disagree, the edge is the thing making policy, not the file.

Item 3

Have you made — and written down — separate decisions for training crawlers and retrieval fetchers?

They are different requests with different consequences. One decision covering both is an accident, not a policy.

Item 4

Do you have a measured raw-versus-rendered visible-text ratio for your main templates?

A measurement, not an opinion about your framework.

Item 5

Do your server logs separate Googlebot, retrieval fetchers and user-triggered agent fetches, with Google-claimed requests verified by reverse DNS?

Client-side analytics record none of this traffic. It exists in logs or it exists nowhere.

Item 6

Is your evaluation window for a shipped technical fix longer than 30 days?

Fixes that land are often judged before the citations that prove they landed.

Your matrix (optional)

Item 4 will not be scored unless you supply both the raw and the rendered figure. A band without a measurement is exactly the thing this check exists to refuse.

Your result
Read the method
What you are being scored against

Our own numbers, from a live page one and our own estate

Five AI fetcher user-agent strings against the seven domains ranking for a head term, re-run four minutes later with the same 35 statuses. One country, one day, seven domains.

MeasuredWhat it is
13 of 35 refused (37%)AI-fetcher requests refused across a live page oneFive AI fetcher user-agent strings against the seven domains ranking for a head term. Every refusal was an HTTP 403. The matrix was re-run four minutes later and returned the same 35 statuses. · Indexable, 9 September 2026, n = 35 fetches
0 of 7Domains whose robots.txt disallowed any AI agent tokenNot one of the seven published a disallow. Every refusal contradicted the file the audit would have read. · Indexable, 9 September 2026, 7 robots.txt files
40 of 40 servedOur own estate's answer to the same testTen user-agent strings against four URLs. Nothing refused — and we still scored 33 on the readiness scale below. Serving the request is the easy half. · Indexable, 9 September 2026, n = 40 fetches
33 of 100Our own composite agent-readiness score, on Indexable's own scaleDiscoverability 67, content accessibility 0, bot access control 33, protocol discovery 33. This is our scale, not a vendor dashboard score. · Indexable, 9 September 2026
median 6.81 days, P90 37.10 daysHow long citations lag publicationThis is the reason a 45-day evaluation window beats a 30-day one: at 30 days you are reading the lag, not the result. Third-party measurement, not ours — and we hold it second-hand, so treat it as directional. · Profound, 2026, as reported in LinkedIn's 2026 AI Search Visibility guide

A 403 to a claimed user-agent string is evidence about edge policy, not vendor intent. Record it as “refused a request presenting as GPTBot”, never as “blocks <vendor>”. One day, one country, seven domains — a case study, not a law.

The matrix takes an afternoon. The argument about it takes a quarter.

An AI search audit returns the status matrix, the robots.txt comparison and the raw-versus-rendered ratio per template — with the edge tickets separated from the SEO tickets, because they belong to different teams.

Get an AI search audit

Copied ✓