VERIFYRAG: A Dual-Layer Hallucination Detection Framework Combining Semantic Consistency Scoring and NLI-Based Evidence Grounding For Open-Domain LLMS

Abstract

Large Language Models (LLMs) are being used more and more in knowledge-intensive tasks, yet they often provide outputs that are factually inaccurate with high confidence, a phenomenon known as hallucinated content. Current methods have little practical utility because they either address detection and mitigation separately or rely on costly proprietary APIs. VerifyRAG, a lightweight, open-source pipeline for detecting and mitigating hallucinations that incorporates a unique dual-layer verification architecture, is proposed in this study. Semantic consistency is used in Layer 1. Semantic alignment between LLM-generated replies and retrieved contextual data is scored using Sentence-BERT. Using DeBERTa-v3, Layer 2 employs Natural Language Inference (NLI)-based Evidence Grounding to categorise each assertion as Entailed, Neutral, or Contradicted in relation to retrieved documents. When the risk of hallucinations is high, a confidence scoring engine that fuses the two layers automatically corrects the response.The system is tested using the TriviaQA, HaluEval, and FEVER benchmarks and is based solely on open-source models. The findings of the experiment show a considerable decrease in the rate of hallucinations while preserving competitive response quality, providing a repeatable and economical approach for the implementation of LLM in the real world.