Adaptive NLI-Driven Claim Verification with Statistical Decision Modeling for Low-Latency Hallucination Reduction in Large Language Models.
TL;DR
A lightweight two-step claim verification framework that decomposes LLM responses into atomic factual claims and independently verifies each extracted claim against a separately generated reference produced through an isolated factual recall prompt, showing consistent performance across the evaluated benchmarks without requiring model retraining.
Abstract
Large Language Models (LLMs) exhibit a critical tendency to generate factually incorrect yet linguistically fluent outputs - a phenomenon termed hallucination - which poses serious risks in precision-critical applications. Existing mitigation strategies, including retrieval-augmented generation and self-consistency sampling, either introduce substantial inference latency or depend on external knowledge infrastructure, limiting their applicability in real-time deployments. This paper proposes a lightweight two-step claim verification framework that decomposes LLM responses into atomic factual claims and independently verifies each extracted claim against a separately generated reference produced through an isolated factual recall prompt. Although the generator and verifier share the same underlying language model, separating response generation from factual recall reduces direct response conditioning and mitigates confirmation bias during verification, using Natural Language Inference, and applies an adaptive statistical threshold - defined as τ = µ + kσ over the NLI confidence score distribution - to selectively correct only contradicted claims. Unlike prior NLI-based methods that rely on fixed decision boundaries, the proposed framework dynamically adapts its verification threshold to the confidence distribution of each response, showing consistent performance across the evaluated benchmarks without requiring model retraining. Evaluated on TruthfulQA and FEVER, the framework reduces the hallucination rate from 28% to 9% on TruthfulQA - a 67.9% relative reduction - while incurring only 160 ms of additional latency over the baseline LLM and outperforming SelfCheckGPT and FActScore in hallucination detection accuracy. These results indicate that the framework can provide a favorable balance between factual reliability and response latency on the evaluated benchmarks, while further validation across domains and deployment settings is needed.