Preprint

SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems

Julia Belikova Rauf Parchiev Mikhail Filimonov Konstantin Polev Andrey Savchenko Maksim Makarenko

Jul 2026 · 0 citations · 32 references

Computer Science

TL;DR

A unified toolkit and interactive web UI for detecting contextual hallucinations in retrieval-augmented, agentic, and memory-grounded LLM systems, and as a faithfulness gate within long-term memory systems is demonstrated.

Abstract

SIRIN (Semantic Inconsistency Recognition and Inspection Nexus) is a unified toolkit and interactive web UI for detecting contextual hallucinations (fluent, plausible responses unsupported by the provided evidence) in retrieval-augmented, agentic, and memory-grounded LLM systems. SIRIN unifies three detector paradigms (representation probing, uncertainty estimation, and judge-style verification) and the complementary task of pre-generation query answerability under one interface, configuration system, and evaluation pipeline, supporting response- and span-level inspection in both white-box and black-box settings. The web UI enables live analysis of user-supplied context-query-answer triples through hallucination scores, unsupported-span highlighting, and side-by-side detector comparison, with a lightweight plug-in design for adding new detectors. We demonstrate SIRIN on hallucination detection, query answerability, and as a faithfulness gate within long-term memory systems. The source code is publicly available at https://github.com/sb-ai-lab/SIRIN.

arXiv Semantic Scholar

View source

Similar papers

Preprint Aug 2026

Decomposed Entailment for Factuality Checking and Hallucination Detection

HallDetect, a lightweight, reference-free, and black-box framework for hallucination detection, is presented, a lightweight, reference-free, and black-box framework for hallucination detection that is evaluated not only on summarization but across a broader range of source-grounded generation settings.

Achir Oukelmoun, N. Semmar, Gaël de Chalendar · 0 citations

Open access Jul 2026

Disentangling Faithfulness Hallucinations in Retrieval-Augmented Generation: A Systematic Benchmark and Analysis

This work analyzes three small-to-mid-sized open-weight LLMs in combination with multiple retrieval strategies, including sparse, dense, and hybrid approaches, as well as score-based and rank-based fusion techniques, enabling a comprehensive assessment of retrieval-generation interactions.

C. Mala, Gizem Gezici, Fosca Giannotti · Machine-mediated learning · 0 citations

Conference Jul 2026

HaloGuard: A Practical Hybrid Framework for Hallucination Detection in Retrieval-Augmented Generation Systems : Combining Multi-Judge Agreement, Uncertainty Quantification, and Lexical Features Without Embedding Dependencies

The issue of hallucination detection for Retrieval-Augmented Generation (RAG) systems is an ongoing reliability challenge that arises with the scale up of LLM deployment. We introduce the HaloGuard: a systems-integration study that integrates three types of signals, multi-judge LLM consensus, uncertainty quantification...

Harshath R, G. S, S. Thangavel et al. · 2026 International Conferenc... · 0 citations

Book Open access Jul 2026

Numerical Hallucinations in Retrieval-Augmented Generation: Detection and Analysis

These findings highlight numerical fabrication as a critical gap in current hallucination detection approaches and recommend the need for specialized, number-aware methods in RAG systems.

S. Singha Roy · Annual International ACM SIG... · 0 citations

#natural language process... Jul 2026

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

GASP, a span-level detector that scores each answer sentence by how strongly its likelihood depends on the retrieved evidence, is introduced, showing GASP is best suited to outputs constructed from the retrieved context rather than answers recoverable from parametric knowledge.

M. Bouke · arXiv.org · 1 citation

Review Open access 2026

Hallucination Is Not One Thing: A Two-Axis Taxonomy for Structured Diagnosis in Generative AI

A concise two-axis framework that integrates an “intrinsic-extrinsic” distinction in source attribution introduced by Ji et al. with a “faithfulness-factuality” distinction in contextual grounding surveyed is presented, yielding four clearly defined hallucination types applicable across tasks, modalities and architectu...

Misbah Khan, Preston Billion-Polak, T. Khoshgoftaar · IEEE Access · 0 citations