Citevio — Field Data on Dental AI Visibility
Citevio — Field Data on Dental AI Visibility
Citevio measures how U.S. dental practice websites appear to AI crawlers and publishes the raw counts as open data. Web: https://citevio.com contact: contact@citevio.com - 30 N Gould St, Ste 45178, Sheridan, WY 82801, US.
Blog By:
Citevio
Citevio

Your robots.txt welcomes AI crawlers. Your server may be turning them away anyway.

8/7/2026 1:16:00 PM   |   Comments: 0   |   Views: 36
Short answer: of 499 U.S. dental practice websites whose robots.txt explicitly allows AI crawlers, 14.4% were refused by their own server anyway when we asked for the homepage as an AI crawler on 22 July 2026. Sixty-nine of those sites returned an outright block status such as 403 or 429, and three returned a bot challenge page. In the same run, only 6 of those 499 sites refused Googlebot.

So the rule your website publishes and the rule your server enforces can be two different rules — and the gap falls almost entirely on the crawlers that feed AI answers.

If you have ever had someone check your robots.txt, been told "you're fine, nothing is blocked," and still wondered why assistants never seem to know your practice exists, this is one of the places to look. It is not the only place, and I will be equally direct about that further down.

What exactly did we measure?

We took a frame of 5,019 U.S. dental practice domains whose robots.txt does not block AI crawlers at the root, drew 550 of them at random, and requested the homepage of each one with seven different crawler identities. After dropping the sites where a plain browser request also failed — those tell you nothing about bot treatment — 499 sites remained. All of it on one day: 22 July 2026.

Every one of these 499 sites had already said "yes" in writing. We were only checking whether the server agreed.

Refused by, out of the 499 sites: ClaudeBot 46 sites (9.2%) · GPTBot 40 sites (8.0%) · OAI-SearchBot 17 sites (3.4%) · PerplexityBot 10 sites (2.0%) · ChatGPT-User 7 sites (1.4%) · Bingbot 6 sites (1.2%) · Googlebot, the control, 6 sites (1.2%).

Read the last two numbers against the first two. Forty-six of these sites turned away ClaudeBot; six turned away Googlebot. That is not a general bot wall. Something is treating these crawlers as a separate category.

Why does this happen?

I do not know, and our data cannot tell you. We measured what servers did, not why they did it. Anyone who tells you the cause with confidence is guessing, and I would rather hand you a clean number than a tidy story.

What I can tell you is what the data rules out. In every one of these 499 cases the practice's own published rule said yes — that is how the sample was built. So whatever is saying no is sitting somewhere other than the file the practice controls. Which layer it is — hosting, a security product, a CDN, a plugin — we did not measure, and I am not going to guess on your behalf. It is a testable question, though, and the person who can answer it for your site is whoever runs your hosting. There is a script for that conversation below.

Is this the same thing as "dentists are blocking AI"?

No, and mixing the two is how bad numbers spread.

"How many practices block AI crawlers in robots.txt" is a different question, with a different sample and a different denominator. It has its own answer and I will write that one up separately rather than blur two measurements into one headline here.

This post is narrower and, I think, more useful: among practices that already allow AI crawlers, how many are being refused anyway? Those practices believe the box is ticked. For 14.4% of them, on the day we measured, it was not.

How do I check my own practice's site? (about 10 minutes)

Nothing here needs a vendor, including me. Three steps.

Step 1 — read your own file. Open yourpractice.com/robots.txt in any browser. Look for GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, ChatGPT-User and Google-Extended. If you see Disallow: / under any of those names, that one is blocked on paper. If you see nothing at all, or only User-agent: * with no Disallow: /, you are permissive on paper — which is exactly the group we measured.

Step 2 — test what the server actually does. This is the step almost nobody takes. Send your hosting provider or web person this line and ask them to run it and send you the first line back:

curl -A "GPTBot" -sI https://yourpractice.com/ | head -1

Then the same with ClaudeBot and PerplexityBot. 200 means you were served. 403, 429 or 503 means you were refused. To make the result meaningful, ask them to run it once more with Googlebot — if Googlebot gets a 200 and the AI crawlers get a 403, you have found the same gap we measured across 499 sites.

Step 3 — ask the right question. If the answers disagree, the question for your host is literally this: "Is there a bot-management, firewall or CDN rule on our account that treats AI crawlers differently from Googlebot? If yes, was it a default we inherited, and can we see it?" Ask for the rule itself rather than a reassurance. A setting you can read is a setting you can decide about; "don't worry, you're fine" is not.

What this does not mean

I would rather lose the argument than oversell the finding, so:

Fixing this does not mean an assistant will start recommending your practice. Being readable is a precondition, not a cause. There is no lever anyone can pull that guarantees a named mention, and anyone selling you one is selling you something I cannot support with data.

Having no robots.txt at all is not a block. RFC 9309, the standard that defines the file, states that if a server indicates the robots.txt file is unavailable, a crawler "MAY access any resources on the server." What a missing file costs you is control, not access.

This is one day and one sample. 22 July 2026, 499 sites, U.S. dental practices. It does not describe your site, your city, or every day of the year. It tells you the question is worth asking about your own site — nothing stronger than that.

FAQ

Is 14.4% a lot?
Compare it to the control: 1.2% of the same 499 sites refused Googlebot. Both numbers come from the same run, the same day, the same sites. Whether that gap matters to you depends on how much you think assistants will matter to how patients find dentists — but the gap itself is not a rounding error.

My site is on a big platform. Am I safer or less safe?
We did not measure blocking by platform, so I will not guess. It is a reasonable question to put to your platform directly, and their answer should be a rule you can see rather than a reassurance.

Does robots.txt affect Google rankings too?
Different question, different mechanism, and outside what this dataset measured. Everything above is about whether a crawler is served the page at all.

Where can I see the raw data?
All of it — the counts, the method, and the places where our own measurement failed — is published as open CSVs under CC BY 4.0 at citevio.com/data. I am linking it so the numbers above can be checked or contested rather than taken on trust; anyone may republish them, including commercially, with credit and the scan date.

About the author and disclosure

Muhammed Veysel Erin is the founder of Citevio, which measures how U.S. dental practice websites look to AI crawlers and publishes the raw counts as open data. Disclosure: I have a financial interest in Citevio. Citevio is a vendor, not a dental practice. No product or service is being offered in this post, and nothing above requires one — every check described here you or your web person can run yourselves.

The dataset behind this post — 499 homepage requests made with seven crawler identities on 22 July 2026 — along with two larger datasets (527 practice websites read field by field in June 2026, and 6,497 robots.txt files read in July 2026) is published as open CSVs under CC BY 4.0, including the method and the known limitations, so that anyone can check or contest the numbers above.
Category: Health, Wellness and Ergonomics, Marketing, Public Health
You must be logged in to view comments.
More In Health, Wellness and Ergonomics
Ergonomics for the Dental Office
Ergonomics for the Dental Office
Dr. Leslie Hovda
CE Credits: 1.75
How Do You Actually Walk in Happy To Work Every Day?
How Do You Actually Walk in Happy To Work Every Day?
October 2024
Product Profile: Vanguard Dental Unit
Product Profile: Vanguard Dental Unit
March 2024
The Top 10 List for an Emergency Kit: Color Coding
The Top 10 List for an Emergency Kit: Color Coding
Dr. Dan Pompa
CE Credits: 2
More In Marketing
2023 Dental Townie Choice Awards
2023 Dental Townie Choice Awards
December 2023
Google Analytics 4: What Dentists Need To Know Now
Google Analytics 4: What Dentists Need To Know Now
Dr. David A. Wank
CE Credits: 1.5
Howard Speaks: Staying On Track in 2024
Howard Speaks: Staying On Track in 2024
by Dr. Howard Farran, DDS, MBA
February 2024
What to Post: 25 Essential Social Media Posts
What to Post: 25 Essential Social Media Posts
Dr. John Syrbu
CE Credits: 1.5
More In Public Health
Better Behavior
Better Behavior
by Dr. William Jacobson
February 2025
Dental Industry News
Dental Industry News
October 2023
Dental Waterline Treatment
Dental Waterline Treatment
July 2025
Product Profile: FastCheck15
Product Profile: FastCheck15
June 2024
Total Blog Activity
997
Total Bloggers
13,451
Total Blog Posts
4,671
Total Podcasts
1,788
Total Videos
Dentaltown Magazine
July 2026
Dentaltown Magazine July 2026 Read Current Issue Read Archived Issues
Sponsors
Townie Perks
Townie® Poll
What do you use to take routine X-rays?