This work analyzes three small-to-mid-sized open-weight LLMs in combination with multiple retrieval strategies, including sparse, dense, and hybrid approaches, as well as score-based and rank-based fusion techniques, enabling a comprehensive assessment of retrieval-generation interactions.
Abstract
Faithfulness hallucinations, where large language models generate outputs unsupported by retrieved evidence, remain a central challenge for trustworthy AI. We present a systematic empirical evaluation of faithfulness in retrieval-augmented generation (RAG) systems using two benchmark datasets, HotpotQA and HaluBench, covering both multi-hop reasoning and single-hop hallucination detection. We analyze three small-to-mid-sized (2B-8B) open-weight LLMs in combination with multiple retrieval strategies, including sparse, dense, and hybrid approaches, as well as score-based and rank-based fusion techniques, enabling a comprehensive assessment of retrieval-generation interactions. By disentangling retrieval and generation errors, we characterize how different pipeline components contribute to hallucinations in RAG systems. Our analysis provides actionable insights and practical evaluation protocols, highlighting the critical role of robust retrieval and careful system design. These findings offer a benchmarking-oriented perspective for developing more reliable and faithful RAG systems within evaluated model scales.
These findings highlight numerical fabrication as a critical gap in current hallucination detection approaches and recommend the need for specialized, number-aware methods in RAG systems.
S. Singha Roy· Annual International ACM SIG...· 0 citations
This review provides systematic theoretical support for industrial RAG model selection and optimization and summarizes existing research gaps, including lightweight deployment and multimodal expansion, and proposes future research directions for trustworthy RAG systems.
Shujing Liu· Applied and Computational En...· 0 citations
A unified toolkit and interactive web UI for detecting contextual hallucinations in retrieval-augmented, agentic, and memory-grounded LLM systems, and as a faithfulness gate within long-term memory systems is demonstrated.
Julia Belikova, Rauf Parchiev, Mikhail Filimonov et al.· 0 citations
Recent inference-time hallucination mitigation methods for large vision-language models (LVLMs) report strong gains on hallucination benchmarks. However, it remains unclear whether lower hallucination scores reflect improved multimodal grounding or more conservative generation. We evaluate six mitigation methods across three LVLMs and four benchmarks, including hallucination-focused evaluation and the diverse capability benchmark MMStar. Our analysis reveals two consistent patterns. First, hallucination reduction is often coupled with reduced informativeness: methods that lower hallucination rates also reduce object recall, visual coverage, or response detailedness. Second, improvements on hallucination benchmarks do not reliably transfer to broader multimodal capabilities, with methods showing inconsistent or degraded performance on fine-grained perception and reasoning tasks. Our findings suggest that current evaluation protocols may overestimate progress by rewarding conservative generation. We argue that hallucination mitigation should be evaluated as a faithfulness--informativeness--capability trade-off rather than through hallucination scores alone.
Mehrdad Fazli, Sina Mansouri, Mohit Marvania et al.· 0 citations
HallDetect, a lightweight, reference-free, and black-box framework for hallucination detection, is presented, a lightweight, reference-free, and black-box framework for hallucination detection that is evaluated not only on summarization but across a broader range of source-grounded generation settings.
Achir Oukelmoun, N. Semmar, Gäel de Chalendar· 0 citations
By unifying four hallucination dimensions with paired question design, KnowHal addresses an important gap in existing evaluation frameworks and enables a more comprehensive assessment of hallucinations in MLLMs.
Ruihan Li, Jiyang Tan, Kailin Jiang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
