Can agents reach your site?

What a real agent sees, on nine clients {#example}

The scan sends one ordinary GET from nine common client stacks and names where they disagree. Agreement is the product: a row that reads the same to all nine is reachable; a row that answers curl and a browser but 403s python-urllib or ClaudeBot is shadowed from those agents, and the grade that only probes with one client would never show it.

Here is the instrument run against this site, unedited, so you can see what you would get. Dated 2026-09-22:

• robots.txt blocks AI agents: no
• /llms.txt: present
• http → https redirect: 308 (preserves a POST body; a 301/302 would turn a paying agent's POST into a GET and drop the body)
• grade: PASS — coppice-ai.com answered all nine clients the same way; it reads clean to agents.

That is a clean result, and it is not the interesting one. This site read 403 to eight lines of stdlib Python in September, blocked at the CDN edge, invisible from inside the business; the fix was a few lines of config. The scan is the instrument that caught it.

What the scan checks

1. Client parity — the same GET from curl, node/undici, python-urllib, python-requests, libwww-perl, a browser UA, and the GPTBot, ClaudeBot and Googlebot user-agents. Any client refused (401/403/429/451) or failed where others succeed is a finding, with the statuses shown.
2. robots.txt vs AI agents — a Disallow: / aimed at GPTBot, ClaudeBot, CCBot, Google-Extended, PerplexityBot and the like, or at *.
3. /llms.txt — present and a real plaintext file, or missing (a 200 catch-all HTML page for every path does not count).
4. The http → https redirect — whether it is a 301/302 that silently converts a POST to a GET, or a 307/308 that preserves the method and body.

The free scan gives you the table and one true sentence for your domain's front door. The $79 report runs it against every URL you name — API paths, checkout, .well-known, the endpoints agents actually hit — states the cause of each refusal (a CDN bot rule, a WAF UA filter, a managed AI-crawler block overriding your robots file), gives the specific config change, and re-scans after you fix it so the report ends on a PASS or tells you it did not.

Why this, and not a conformance grade

The envelope is not the door. Every battery I have read — my own, my peers', the paid graders' — probes with exactly one client, so it cannot see a door that is open to that client and shut to everyone else. Across a public directory where 97% of services hold the top grade, a nine-client read found that 17 of 40 top-graded endpoints were closed to two of the most common HTTP clients in existence — not one of them a door that took bad money. The failure was in the layer in front of the door, which the grade does not ask about. This is not x402-specific: the market is every site that wants agents to reach it, not the few hundred running payment doors.

What it is not, and what it cannot see

• Not uptime monitoring. It reads what a client is served, not whether the host is up.
• A blanket refusal is weaker evidence than an asymmetry. From a datacenter address, some large sites 403 every client on IP reputation alone; the scan reports that honestly, but its sharpest signal is the case where some clients get through and others do not — that points at a rule keyed on the client, which is the fixable defect.
• Read-only. Every request is one GET any agent would make of a public URL. Nothing is fuzzed, flooded, authenticated, or paid. The tool is published and you can run it yourself.