Conference
NLPCC
IntegriRef: A Five-Layer Bayesian Framework for Cross-Domain Reference Integrity Verification
Zhaohui Wang
CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC), 2026
// ABSTRACT
LLMs frequently fabricate bibliographic references, generating plausible but false DOIs, authors, venues, and citation metadata. Existing tools mostly verify whether a DOI resolves, missing subtler integrity failures such as metadata chimeras, citation misrepresentation, and citation-ring manipulation.
IntegriRef is a five-layer pipeline that queries 62 registry adapters and aggregates 21 typed signals with Bayesian likelihood-ratio scoring. Its operational core is multi-registry existence checking (L0) fused into a calibrated risk tier (L4); intent, NLI and citation-graph layers are targeted escalation modules, and we report what each is and is not worth. On two externally labelled splits (Retraction Watch, Crossref, PubMed), IntegriRef achieves 99.9% recall on 1,215 hallucinated references and 98.3% recall on 5,488 retracted papers (FPR 4.6%, 4.2% on matched controls). On a 58-case golden set it reaches 85% hallucination recall at 0% FPR, while the highest-recall parametric LLM (gpt-oss:120b) reaches 95% at 20% FPR, and adding a Crossref tool degrades both glm-4.6 and deepseek-v4-pro.
Our analysis locates where verification succeeds and fails: registry checks catch most fabrication, but ID-resolving chimeras need cross-field validation. Raw posteriors rank well yet are poorly calibrated (ECE 0.36 to 0.003 after Platt scaling); empirical likelihood ratios differ from hand-set values by 1-2 orders of magnitude, and 9 of 21 signals stay empirically unvalidated. Fusion refines severity in every multi-signal case but flips no escalation decision - L0 makes that call. We trace a year-only chimera blind spot (0/60) to a verdict-downgrade rule and fix it (60/60 CRITICAL at 0/60 FPR). Evaluated at N>200 rather than 40, the intent and NLI layers prove markedly weaker than small splits suggested.
IntegriRef is a five-layer pipeline that queries 62 registry adapters and aggregates 21 typed signals with Bayesian likelihood-ratio scoring. Its operational core is multi-registry existence checking (L0) fused into a calibrated risk tier (L4); intent, NLI and citation-graph layers are targeted escalation modules, and we report what each is and is not worth. On two externally labelled splits (Retraction Watch, Crossref, PubMed), IntegriRef achieves 99.9% recall on 1,215 hallucinated references and 98.3% recall on 5,488 retracted papers (FPR 4.6%, 4.2% on matched controls). On a 58-case golden set it reaches 85% hallucination recall at 0% FPR, while the highest-recall parametric LLM (gpt-oss:120b) reaches 95% at 20% FPR, and adding a Crossref tool degrades both glm-4.6 and deepseek-v4-pro.
Our analysis locates where verification succeeds and fails: registry checks catch most fabrication, but ID-resolving chimeras need cross-field validation. Raw posteriors rank well yet are poorly calibrated (ECE 0.36 to 0.003 after Platt scaling); empirical likelihood ratios differ from hand-set values by 1-2 orders of magnitude, and 9 of 21 signals stay empirically unvalidated. Fusion refines severity in every multi-signal case but flips no escalation decision - L0 makes that call. We trace a year-only chimera blind spot (0/60) to a verdict-downgrade rule and fix it (60/60 CRITICAL at 0/60 FPR). Evaluated at N>200 rather than 40, the intent and NLI layers prove markedly weaker than small splits suggested.
// BIBTEX
@inproceedings{wang2026integriref,
title = {IntegriRef: A Five-Layer Bayesian Framework for Cross-Domain Reference Integrity Verification},
author = {Zhaohui Wang},
booktitle = {CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC)},
year = {2026},
}