Is my dental website blocking ChatGPT?

Your robots.txt file can say yes while your server still says no. That is the answer to "is my dental website blocking ChatGPT?" The public policy file is only the first gate. The homepage request meets a second gate, where the server applies a different rule.

Here is the measured result. In Citevio's live crawler access study on 22 July 2026, 72 of 499 permissive US dental practice websites, or 14.4%, refused at least one AI crawler at the server. That is an access finding, not a promise about being named by AI. This is one day's measurement. Hosting and content delivery rules change without notice, and a repeat on another day will not reproduce this file exactly.

That design starts after the policy question has already been settled. Every site in the final sample allowed AI crawlers in robots.txt. Every homepage also loaded for a plain browser request. We then changed the identity making the request and recorded whether the server treated the crawler identity differently.

Why separate those layers? Because robots.txt is a statement. A server response is enforcement. The statement can be permissive while a firewall, hosting rule, security layer, or another system refuses the request later. The public rule and the delivered page can disagree without either record disappearing.

The control request makes the finding easier to interpret. In Citevio's live crawler access study on 22 July 2026, 6 of 499 permissive US dental practice websites, or 1.2%, refused Googlebot. That gives us a way to distinguish a general bot wall from treatment aimed at a particular identity.

The strongest comparison is narrower. In the Citevio crawler access study on 22 July 2026, 12.6% of those 499 permissive US dental practice websites refused an AI crawler while serving Googlebot normally. The same homepage gave a different answer depending on who was asking for it. This is why a single browser check is weak evidence. It proves only that the ordinary request succeeded. It says nothing about the response delivered to another identity. Preserve the request headers, response status, final URL, and a small body signature so another person can reproduce the comparison. If the evidence cannot be reproduced, the diagnosis is still only a guess.

What caused the difference? We do not know. The study recorded responses, not configuration histories. It cannot tell us whether a host, content delivery network, security product, plugin, or human choice created the rule. Naming one as the cause would turn an observation into a diagnosis our instrument cannot support.

The finding still rules out one explanation. The refusal cannot be explained by the public robots.txt file alone, because that file permitted the crawler by design. Whatever created the refusal acted after that file had already answered.

Now translate that into a practice-level check. Do not ask only whether robots.txt looks open. Ask whether the same homepage is actually delivered when the requester identifies itself differently. A useful comparison needs a baseline and a control, so start with the same three identities.

First, request the homepage as an ordinary browser. If that request fails, the result does not isolate crawler treatment. It could be a general outage, a network failure, or a broken page. Establish that the homepage can be served before blaming the crawler-specific layer.

Second, request the same homepage under an AI crawler identity. Keep the page, the moment, and the request method the same. Record the returned status and the response body. A statement that someone "checked the setting" is not the same as the record you need.

Third, use Googlebot as a control. If the ordinary browser and Googlebot receive the homepage while the AI crawler is refused, the pattern becomes specific rather than general. You still do not know the cause, but you know which layer deserves inspection.

Do not stop at the status code. A challenge page can arrive with a superficially successful response. Compare the page body, content length, and visible title. The question is whether the crawler received the real homepage, so preserve enough of each response to compare the requests rather than the number alone. A useful body check looks for a stable piece of the real page, such as its visible practice name or a known heading. That check also needs a sanity review. A matching fragment does not prove full rendering, and a changed fragment does not automatically prove blocking. Record what arrived before deciding what it means.

Then ask the host for the exact rule that produced the difference. The question is not "are we fine?" It is "which rule handled this user agent, and can we see it?" A rule can be reviewed and changed. Reassurance without a result you can see is not evidence. If support cannot identify the decision point, ask them to reproduce the requests beside you and retain the result. Do not authorize a broad security change merely to make a test pass. The narrow goal is to understand the existing behavior, choose the intended policy, and verify that the live response now matches that choice.

Should every crawler be allowed? That is a business and policy choice for the practice. Some crawlers may be refused intentionally. The measurement does not tell you what your policy should be. It tells you the published choice and the enforced choice should be the same.

Here is the causal limit. We did not measure whether removing a refusal causes an assistant to name, cite, or recommend a practice. We measured whether a homepage was served. We have not measured whether changing this access state changes an AI visibility outcome in a controlled experiment.

Being readable is a prerequisite. It is not a guarantee. An engine may fetch a page and still omit the practice. It may use different sources or rebuild the answer on another run. A successful request removes one obstacle; it does not purchase a result that no vendor can honestly offer.

The same caution applies to time. We did not measure how long any access change takes to appear in an assistant answer, and we did not measure a time-to-citation distribution. A schedule for retesting can be chosen. A duration promise cannot be derived from anything measured in this access study.

So the practical conclusion is small and testable. Read the policy file, then test the delivered page. Keep the ordinary browser and Googlebot controls. If the responses differ, ask for the exact enforced rule. Do not infer a visibility outcome. Confirm what the server actually does.

The aggregated counts, method, and known limitations are published at citevio.com/data. No practice name or domain is published in the aggregate package, and no client data is included. If you want to challenge the denominator or the definition of refusal, you will find the complete record there.