If You Found One Fake Citation, Look for the Others
When AI scaffolds produce a fabricated citation, the fix is not to remove it. The fix is to grep every other document in the project for the same fabrication signature.
Jason Walker
State CISO, Florida
A fact-checker flagged a citation in my literature review chapter. Theriot et al., 2023, a paper supposedly published at a workshop on the economics of information security. The chain ran six paragraphs deep. CrossRef returned nothing. The conference proceedings did not list it. The fabrication was clean enough that it had survived three earlier review passes, but a fourth pass with a fresh agent and a tight scope caught it. I excised the citation, rewrote the dependent paragraphs, and moved on. That should have been the end of it.
It was not.
A week later, while running an inventory audit on the citation library, I noticed the same author surnames in the introduction chapter. Same year. Same fictional paper. Cited as load-bearing evidence in the dissertation's contribution-claim paragraph and again in the reference list. The introduction chapter had been submitted to my chair four days before the literature review fact-check ran. The fabrication was already in a delivered artifact.
This is the part of AI-assisted academic writing that nobody warns you about. Fabrications cluster.
What most people get wrong
The standard fact-check workflow assumes verification is local. You write a draft. You run a verification pass on that draft. You catch the fakes. You submit. The unstated assumption is that each fabrication is independent: a coin flip that the LLM happened to lose on this particular paragraph. If you check enough paragraphs, you catch enough fakes.
That model is wrong. When an LLM produces a fabricated citation in one drafting session, it tends to produce the same fabrication in adjacent sessions. The hallucination is not random. It is statistical, attached to the topic and the prompt context that triggered it. If your session prompts include "agreement testing for FAIR loss components" the model has a recurring tendency to reach for the same fictitious paper, the same fictitious authors, the same fictitious venue. It is not making up a new fake every time. It is reaching for an attractor that lives in the model's distribution.
That means a fabrication in your literature review is a strong predictor that the same fabrication exists in your introduction, your methodology section, your prospectus, your annotated bibliography, and any side notes you generated in the same campaign. Verification that scopes to one document at a time misses everything except the document you happened to verify.
The rule
When a fact-check sweep on AI-generated content finds a fabrication, the next step is not "remove this one." The next step is "grep every other artifact in this body of work for the same fabrication signature."
Concretely: take the surname of the fictitious first author. Take the year. Search every chapter file, every prior draft, every working note, every email thread to your committee, every supplementary document. Check what comes back. If the surname appears anywhere else with the same year, you have a second hit on the same hallucination, and you need to handle it the same way you handled the first one.
The cost of this step is low. A grep across a dissertation directory takes seconds. The cost of skipping it is high. A single fabrication that survives into a submitted document is harder to retract than to remove, and the credibility damage compounds if a committee member finds it before you do.
Evidence from my own work
This came up while I was running a citation verification campaign across two dissertation chapters. The literature review chapter went through a structured fact-check that excised seven distinct fabrications, one of which was the Theriot citation. The introduction chapter had gone through a verification pass earlier in the month, but that pass was scoped to its own bibliography. It did not cross-check against fabrications discovered in later chapters.
The result: same fictional paper, two different documents, only one document caught it. The introduction was submitted with the fake citation in place. The fix has to wait for the next milestone version.
If I had run a cross-document grep at the moment Theriot was excised from the literature review, I would have caught it. The grep takes about three seconds. The fix takes about ten minutes. Instead the fabrication is now part of an evaluated submission, and the cleanup is permanent in the way that submitted artifacts are permanent: you can correct the next version, but the trail does not erase.
What to actually do
Build the cross-document sweep into your AI-assisted writing workflow. Not as an optional polish step. As a hard gate before any "verification complete" call.
The minimum version is one shell command:
grep -rn "<surname>" path/to/project/
Run it the moment a fabrication is identified. Read every match. For each match, decide whether the citation is the same fabrication or a coincidental real source. Excise the matches that are part of the cluster. Rerun the verification on the affected paragraphs. Then declare the sweep complete.
If you are working at scale, automate it. Maintain a list of confirmed-fabricated citations as a project-level artifact. Every verification pass appends to this list. Every commit hooks into a cross-document grep that fails the build if any item in the list still appears in any artifact. The fabrication list becomes a kill list.
The deeper lesson is that AI-assisted writing is not document-scoped. Your prompts cluster, your sessions cluster, your hallucinations cluster. Treat the project as the unit of verification, not the document. The grep is cheap. The citation that survives is not.