Uncertainty-Aware Structural Attention Features for Hallucination Detection in LLMs
Large Language Models frequently generate responses that appear fluent and coherent but may contain factually incorrect information, a phenomenon known as hallucinations. Hallucinations have emerged as a significant barrier to the effective application of Large Language Models. Some studies suggest that hallucinations can be detected by analyzing structural properties of attention maps. However, such methods primarily focus on attention structures and their spectral features, whereas uncertainty, randomness, and other statistical measures can also be leveraged to detect hallucinations. In this work, we propose a method for detecting hallucinations using spectral and statistical features extracted from attention maps. We compute eigenvalues of the Laplacian matrix from each attention map to capture spectral features and compute statistical measures for every attention head. The spectral and statistical features are used as a feature vector for hallucination detection. We conducted experiments on the HaluEval dataset across multiple language models, and the results show that jointly modeling structures and uncertainty-aware characteristics leads to more robust hallucination detection.