You Don't Ask the Liar If He's Lying

How One Hallucination Becomes the Whole Corpus

In December 2024, a computer vision team posted a preprint to arXiv. NFL-BA, by Beltran et al. (arXiv:2412.13176v1). Real authors. Real research. Subject matter: endoscopic SLAM. Unrelated to law in every respect.

The v1 bibliography contained one fabricated citation: a phantom CVPR 2022 paper that did not exist. The authors corrected it in later versions of the preprint. But v1 had already been pulled into foundation model training data.

Twelve months later, that exact fabricated citation showed up in an accepted paper at NeurIPS 2025, the most selective AI conference in the world. Researcher Samar Ansari (University of Chester) traced it and named the mechanism: Contamination Inheritance (arXiv:2602.05930).

That happened in a niche academic field. The number of people who care about endoscopic SLAM is small. The contamination loop still propagated.

Yesterday, Anthropic launched Claude for Legal.

What Just Happened on May 12

The Claude for Legal launch is the largest single concentration event in legal AI to date. The package: 12 practice-area plugins, more than 20 MCP connectors into law firm software, a deep Microsoft 365 integration that puts Claude inside Word, Outlook, Excel, and PowerPoint as a context-carrying agent, and named deployments at Freshfields, Quinn Emanuel Urquhart & Sullivan, Holland & Knight, and Crosby Legal. Harvey ($11 billion valuation), Legora ($600 million Series D), Solve Intelligence, Eve, and Thomson Reuters' CoCounsel (which Thomson Reuters has rebuilt on Anthropic) all run on Claude underneath.

Outside the legal vertical, the scale is larger still. Anthropic disclosed more than 300,000 business customers in October 2025. Claude is now available natively on both AWS (through Claude Platform on AWS, also launched this week) and Google Cloud (through Vertex AI). The legal AI consolidation announced May 12 is one wedge of a broader concentration in which a single foundation model has become the underlying generator for a substantial portion of enterprise AI work.

Why That Matters

A monoculture amplifies contamination. This is well-established in systemic risk theory. When every institution uses the same risk model, errors do not get cross-checked across the system; they cascade simultaneously. The 2008 financial crisis is the canonical example. The principle generalizes.

Apply it to legal AI as of yesterday.

Before May 12, the legal AI market was fragmented across multiple foundation models. Errors from one model had at least an informal chance of being caught by users of a different model writing on the same legal issue. The cross-checking was imperfect. It existed.

After May 12, the cross-checking is gone. If a hallucinated citation enters Claude's training corpus once, it can now reach most of the legal AI stack simultaneously. The hallucination gets reproduced across Harvey, Legora, CoCounsel, Eve, and the Microsoft Word agent at the same time, because they all sit on top of the same generator. The brief gets filed. PACER indexes it. Westlaw indexes it. The brief becomes training data for the next Claude generation. The same hallucination propagates with greater statistical weight than before, into a generator that already learned it once, in front of a customer base of 300,000+ business accounts, distributed through both major US cloud providers.

The Contamination Inheritance loop that produced one fake citation in one academic paper took twelve months, one preprint, one fake citation, and a training cycle. The legal version of that loop now runs through every brief filed in every federal and state court in the United States, on one underlying model, distributed everywhere. Each cycle of training on its own output pulls the legal corpus further from the underlying primary sources.

The Empirical Proof That Vendor Governance Does Not Catch This

One year ago today, on May 13, 2025, a federal magistrate judge struck paragraph 9 of an Anthropic data scientist's declaration in Anthropic's own active litigation because Claude had generated a fabricated citation that Latham & Watkins' manual cite-check did not catch. Concord Music Group, Inc. v. Anthropic PBC, Case No. 5:24-cv-03811-EKL (SVK) (N.D. Cal.), 2025 WL 1482734.

With Anthropic's full safety stack live. With Anthropic's own employee as declarant. With one of the world's largest law firms running the verification. In the company's own defense. The court was the entity that caught the hallucination. Not Claude. Not Anthropic. Not Latham.

That is the empirical baseline.

The Auditor Principle

There is a basic principle in auditing: the audited entity does not design its own audit. You do not let the company being investigated for fraud write its own internal controls. You do not ask the liar whether he is lying. You do not let the liar build the workflow that decides whether he was lying.

This is not a rhetorical flourish. It is the principle that produced Sarbanes-Oxley after Enron, that requires SOC 2 audits to be performed by independent firms, that prohibits Big Four accounting firms from also serving as paid consultants to the same audit client on substantive matters.

This is not an argument that Anthropic acts in bad faith. The argument does not require it. A vendor cannot independently audit its own product. Even with perfect intent, the verification answers the questions the generation knows how to pass. The blind spots are constitutive of the worldview that produced the product. The Concord record demonstrates this empirically in the legal context with Anthropic itself as the test case.

What Independent Governance Means

Three concrete implications, none of which require buying additional AI products.

The Bottom Line

One preprint contained one fabricated citation. That citation propagated through training data into peer-reviewed publications at the world's most selective AI conference twelve months later. The mechanism is documented and named. Yesterday, the underlying generator for most major legal AI products converged on a single model that already serves 300,000+ business customers through both major US cloud providers, and one of the bundled plugins is called AI Governance Legal.

The choice in front of small and mid-sized firms is not whether to use AI. The choice is whether the verification layer is independent of the generator and the generator's vendor.

Nothing in this article constitutes legal advice. Jurisdiction-specific ethics analysis is required before relying on any practice described here.

---