Lightweight Multi-Agent Verification Framework for Trustworthy RAG to Reduce LLM Hallucinations

Abstract

Although Large Language Models (LLMs) are mostly effective, they sometimes produce outputs that are not necessarily accurate, and Retrieval-Augmented Generation (RAG) offers a solution to the problem by getting the model to read some materials before producing the answer, but RAG may not be sufficient to resolve the issue completely. The retrieved sources may be anything from irreverent and contradictory to obsolete and unreliable. As a part of a standard RAG system, the verity of the retrieved information is not checked before giving the input to the language model. The present paper presents the results of ten studies dealing with multi-agent and multi-level verification of data, facts verification through hallucination detection, the process of responding to the debate, agent coordination, knowledge-graph and hypergraph-based retrieval, and the use of RAG-based reliable systems. The major issues include high costs of implementing the most effective method and the fact that in many cases verification is done post factum. To solve this problem, we proposed a simple model where only a few small agents check the relevance, reliability, and up-to-dateness of the quoted statements.The delivery of the passages occurs through a one-time single call, while there is no reply from the model in case all passages failed to pass the test. The project is more theoretical in nature, as we provide the description of the architecture, scoring model, algorithm, and evaluation plan, leaving the practical part of the project for the future.