Tackling LLM Hallucination with Abductive Reasoning
TL;DR
Evaluation of the method for checking whether such explanations are supported by the available evidence shows that incorporating this structure improves the detection of hallucinated reasoning compared with the same detection approach without structural information.
Abstract
Large language models can produce convincing explanations that rely on invented facts, unsupported assumptions, or overlooked alternatives. This paper presents a method for checking whether such explanations are supported by the available evidence. The method identifies the assumptions needed to support a model's answer, checks for contradictions, and compares the answer with alternative explanations that require fewer unsupported assumptions. It also considers how an explanation is organized, distinguishing its central claims from supporting details and examining how statements support or challenge one another. Evaluation on four datasets shows that incorporating this structure improves the detection of hallucinated reasoning compared with the same detection approach without structural information. Additional experiments examine the contributions of explanation structure, assumption checking, and comparison with alternatives. The findings suggest that checking both the evidence behind an answer and the organization of its explanation can help detect and explain failures in language-model reasoning.