How the grades work
Rubric v0, scoring rules v0.1. The full rule definitions and change history live in the public code repository.
In plain words
Every insurer selling plans on HealthCare.gov must publish its full list of doctors and therapists as a public data file and keep it current. That duty comes from a federal rule, 45 CFR 156.230(b), enforced by the Centers for Medicare and Medicaid Services (CMS). We download every one of those files each month and archive them.
Then we check every entry for problems a patient would hit: phone numbers like 999999999, update dates from the year 1900, providers listed hundreds of miles from the plan's counties, and provider ID numbers the federal government has retired. Every problem we find becomes one row of evidence, tied to an archived copy of the insurer's file. The rows add up to a grade for each plan in each county.
Two limits. The grades are computed from files, not phone calls. A good grade means the published list is consistent and current, not that an office will answer. And the score never judges care quality or any individual clinician. It judges the insurer's published data.
The ten checks
Problems with the whole file
Dead file. The web address the insurer gave CMS returns nothing, or most of the files it points to cannot be read. We cannot audit a file we cannot fetch, so every plan from that insurer is marked X for unauditable instead of scored. An F means the published content is bad. An X means there was nothing to check.
Browser-only access. Some insurers' files load in a web browser but refuse automated tools, which defeats the purpose of a machine-readable file. We note this on the insurer's page. It does not lower the score, because the content, once fetched, is complete.
Problems with individual entries, read straight from the file
Placeholder values. Phone numbers like 999999999, ZIP code 99999, street addresses that literally say "null", and update dates before 2014. An entry like that is unusable exactly as published. This is the heaviest penalty.
Old update dates. Each entry carries a date saying when it was last updated. We flag entries older than 180 days, and more heavily past 365. One caveat is mandatory. Most files stamp one shared date on every entry, so the date behaves like a file-generation timestamp. An old date is a real warning sign. A fresh date proves nothing about any single entry, and we never present a fresh file as verified.
One shared phone number and nothing else. Every phone number on the entry is a number that appears on at least 1% of the file's entries or 50 entries, whichever is larger. We identify whose number it is where we can, because a provider group's own booking line is a smaller problem than a number that reaches nobody. We also check whether the federal registry lists a different direct phone for that provider.
Many addresses on one provider. One individual provider listed at more than 10 street addresses at once, or an organization at more than 25. We skip specialties that legitimately cover many sites, like radiologists.
Out-of-area listings. A provider attached to a plan even though none of the provider's addresses fall inside any county the plan covers, or any neighboring county. These entries add to the provider count without adding anyone a member can reach. They are excluded from county counts and scored at the plan level.
Missing accepting-patients answer. Individual providers are required to say whether they accept new patients. We flag entries that leave it blank or say unknown. Facilities and groups are allowed to omit this field, and we never count that against them.
Entries that disagree with the federal provider registry
Every US provider has a National Provider Identifier (NPI), a 10-digit number issued by the federal government and tracked in a public registry called NPPES. The registry publishes a monthly list of retired identifiers.
Registry status. We flag entries whose NPI is missing, fails its checksum, or appears on the federal retired-identifier list while absent from the active registry. Identifiers are retired for many reasons, including retirement, death, and practice changes. The flag says one thing only: the insurer's current directory carries an identifier the federal registry retired, as of a dated report.
Specialty disagreement. The file lists a provider under a mental health specialty while the registry records only unrelated fields, such as physical therapy. One of the two databases is wrong about this provider. We do not know which one. The flag states both facts.
How checks become a grade
Every grade attaches to one plan in one county. We grade the mental health list separately from the all-provider list, and both appear side by side.
Each listed provider starts clean. Each problem adds a penalty, but penalties on one entry do not simply add up: a badly broken entry counts once, at the weight of its worst problem plus a fraction of the rest. One broken record is never counted four times.
The in-county entries carry 80% of the grade. The plan's out-of-area rate carries the other 20%, because those entries never appear in any county count and would otherwise be invisible.
A registry disagreement can lower a score by at most 20 points. The registry is itself self-reported and lags reality, so we hold that cap until insurers have had one monthly notification and correction round to respond.
We also recompute every score with all penalty weights set equal, as a check that our chosen weights are not driving the story. Where the two versions land in different letter grades, the plan's page shows both. The full data download carries both scores for every plan and county.
Grade bands: A is 90 to 100, B is 80 to 89, C is 70 to 79, D is 55 to 69, F is below 55, and X means unauditable. These cutoffs are our own. They are not yet tied to any official standard or benchmark. Once we see how scores are distributed across all plans, we will adjust the cutoffs and publish exactly what changed.
Small lists get special handling. A county list with 10 to 29 providers is graded with a low-sample badge. A list with fewer than 10 is not graded at all; the small count itself is published as the finding.
Three evidence labels
Every flag carries one of three labels, so a reader always knows how strong the evidence is.
E1: read straight off the insurer's file. Anyone with the archived file gets the same answer. Example: a phone number of 999999999.
E2: read off the insurer's file, through a cutoff we chose. Example: we call an update date stale after 180 days. Every cutoff is published, and the table below shows how the counts move at other cutoffs.
E3: the file disagrees with a federal registry snapshot. Either database could be the stale one. These flags always carry the registry date and the note that identifiers are retired for many innocent reasons.
The cutoffs at other settings
When findings publish on October 26, 2026, this section will show how each count moves at looser and stricter cutoffs.
Known biases
Most files stamp one shared date on every entry. A fresh date therefore proves nothing, and freshness never earns credit.
The file format lets facilities and groups omit some fields. We never count those omissions as defects. Early in this project we nearly published a wrong number by missing that; the correction is part of the public record.
A few hosting companies publish files for many insurers at once. A bug at one host can lower scores for several insurers at the same time, so every insurer's page names its hosting company.
The federal registry is not ground truth. It is another self-reported database with its own lag. That is exactly why registry disagreements are capped at 20 points.
How we treat insurers fairly
- Every rule has a version number, and every published flag records which version produced it.
- Every downloaded file is archived with its web address, download time, response headers, and a cryptographic fingerprint (SHA-256). Claims cite the archived copy, so anyone can re-check them.
- At least 14 days before publication, each insurer's technical contact on file with CMS receives its complete flag export.
- Disputes sent to contact@ghostnetworkwatch.org or filed through the public correction tracker are published verbatim alongside the flags they dispute.
- When a later monthly crawl no longer reproduces a flag, it will be marked resolved with the date. History is retained, not deleted.
- Crawls run monthly, aligned to the federal registry's monthly release. Every page shows its snapshot date, and pages older than 60 days say so.