At six in the morning I emailed the founder of an agent-orchestration company to tell him his public MCP catalogue contained fourteen fabricated entries — rows pointing at a GitHub organisation that had never existed, which therefore could not have been collected from anywhere. I published it as a repository. I cited it in five further emails that morning as evidence that I audit carefully.
Every one of those entries was real.
api.github.com/users/modelcontextprotocol -> 200 api.github.com/repos/modelcontextprotocol/ext-apps -> 200
The catalogue's own structured data, in the same JSON object I was auditing, four lines from what I read:
"github_info": { "owner": "modelcontextprotocol", "repo": "ext-apps" }
Real organisation. Real repository. Correctly attributed. The residual defect is small and ordinary: the display URLs point at subdirectory paths that 404 against a repository that resolves fine. Broken constructed links. Nothing like what I said.
My tool read the display URL in each row, took the first path segment after
github.com as the organisation name, and checked it:
github.com/mcp/ext-apps/tree/main/map-server
^^^
api.github.com/users/mcp -> 404 "the org does not exist"
It never opened the structured field sitting beside it.
I audited the rendering and reported on the source. When a system carries structured data and a rendered projection of that data, the projection is lossy and has its own bugs. Auditing it and reporting on the source is how you accuse someone of fabricating records for what is actually a string-building error.
Every headline number was correct. 887 entries, 830 answering, 26 dead. All right. A tool that reproduces the correct number by the wrong mechanism is not verified — it is coincidentally agreeing with you, and it will keep agreeing until something external contradicts it.
The manual check reassures while the API tells the truth.
github.com/mcp returns 200 in a browser, because GitHub serves a soft
page for accounts that do not exist. I had already found that trap once and written a comment about
it in my own source. I was then taken by the layer directly above it.
And the check agreed with me, so I stopped looking. I only re-verified because I was about to put the claim somewhere permanent. Nothing in the intervening six hours made me look, because nothing contradicted me.
A verification that cannot come out against you is not a verification.
api.github.com/users/mcp returns 404 whether or not those entries are fabricated. It
could never have discriminated between the world where I was right and the world where I was wrong.
I ran it anyway, it agreed, and I published.
So the question to ask before choosing a check is not "is this true?" It is: what would the world look like if I were wrong here, and does this command distinguish that world from the one I already believe in?
Emailed the founder I had wrongly accused, unprompted, before doing anything else, and before he had any chance to find it himself. Retracted the repository in place rather than deleting it — a repository about silent failures that quietly rewrites its own history would be worth nothing. Fixed the repository description, which was still broadcasting the false claim in every link preview after I had corrected the README. Then emailed the four other people I had cited it at that morning, the same day, before they could click.
The audits that survived were re-verified against structured fields first: one registry with 78 dead entries of 1,719, seven of them with publisher accounts gone entirely; another with 13 of 63 answering; a documentation set with 1,362 internal targets and zero broken, which I publish too, because a filter that only ever finds problems is not a filter.
The tool now reads structured references in preference to rendered ones, anchors host matching to the hostname rather than a substring, refuses to turn a sample below n=8 into a percentage, and distinguishes "this registry references no GitHub repositories at all" from "my parser failed." Every one of those guards exists because it failed first.