How Obi checks AI crawler readiness - Obi

Readiness answers a technical question: can the AI engines’ crawlers reach your pages, and can they read the facts that matter on them? Obi saves each Readiness report with the evidence behind every finding.

What Obi checks

Area	What it looks at
Crawler access	Your `robots.txt`, read rule by rule for each AI search and user-fetch crawler and each page checked, with the line that decides
Indexability	Pages that are unavailable, marked `noindex` or `nosnippet`, restricted from Bing answers, or whose canonical points elsewhere
Rendering	Material facts that only appear once a browser runs the page’s JavaScript, or that change when it does
Structured data	Whether the pages carry structured data, and whether it names another host
`llms.txt`	Whether the file exists, is valid, and links to pages that answer
Coverage	How pages were discovered (sitemap or links), how many were checked, and important pages nothing links to

Blocking a training crawler, such as GPTBot or Google-Extended, is reported as your preference, not as a defect. Obi separates the crawlers that power AI search and live answers from those that only collect training data.

Reading a finding

Each finding carries:

a state: pass, fail, unknown or not applicable;
a severity: blocker, major, minor or info;
a confidence: high, medium or low;
its evidence: the address, the time it was observed, how it was fetched, and the quoted lines;
an explanation: what was observed, what it costs you, how to correct it and how to verify the correction.

When a robots.txt rule is the cause, the finding can include a patch with the exact diff and the number of intended and unintended changes across the pages compared. A check that could not run is reported as not run, with its reason. A blocked page or an incomplete check is unresolved, never a clean result.

Where it fits

Obi reads Readiness while it studies your business, and uses its findings when it ranks your priorities. Ask Obi for your Readiness report at any time, or run obi readiness: reading the saved report never starts a new reading.

Next: strategies and outcomes