Adaptive Multi-LLM Hallucination Detection web application

Abstract

LLMs have become more widespread in many critical areas, which has increased the risk of hallucinations, or instances where generated outputs contain false or misleading information. To address this problem, this paper introduces HalluciNet v3.0+, which is an adaptive multi-LLM hallucination detection framework that employs advanced preprocessing, feature engineering, and classification methods for strong factual verification. In the preprocessing stage, the system begins with normalizing raw LLM-generated outputs; using linguistic parsing through spaCy to break up the LLM outputs into structured claims; and then using Transformer-based tokenization for contextual refinement of the structured claims. For the feature extraction step, the system uses both BERT and BGE embeddings to create dense semantic representations of the structured claims and the context and relationship dependencies between the structured claims and the retrieved evidence. A hybrid feature selection is achieved by combining Recursive Feature Elimination (RFE) and attention-weight optimization methods, which provides for the selection of the most discriminative features while minimizing dimensionality. A novel ensemble-based classification solution is implemented using DeBERTa-v3 for Natural Language Inference, XGBoost for structured decision learning, and consensus scoring across multiple LLM sources to produce a final predicted output. A two-step RAG (Retrieval-Augmented Generation) module optimizes retrieval and retrieval based on the quality of the retrieved item in a cross-encoder-based framework while preserving the inherent factual quality of the item's contents. This system is designed to be fully scalable and production-ready and is therefore expected to provide trustworthy enhancements to the use of LLM-driven products.