Large Language Models Hallucinate and How Retrieval- Augmented Generation Mitigates It
TL;DR
It is concluded that RAG should be viewed as a grounding and evidence-access mechanism rather than a guarantee of hallucination-free generation, and applications of RAG are outlined.
Abstract
Large Language Models (LLMs) can generate fluent and convincing responses, but fluency does not guarantee factual correctness. Hallucination occurs when a model produces information that is false, unsupported, or inconsistent with available evidence. This paper reviews why hallucinations arise andexamine Retrieval-Augmented Generation (RAG) as a practical mitigation strategy. The discussion focuses on prediction-based generation, missing or outdated knowledge, ambiguous queries, and the absence of automatic verification. RAG addresses these limitations by retrieving relevant information from an external knowledge source and supplying it to the generator as contextual evidence. The paper describes the major stages of a RAG pipeline, including document ingestion, chunking, embeddings, vector storage, retrieval, context augmentation, and answer generation. It also examines the conditions under which RAG can fail, including poor retrieval, incomplete knowledge bases, unreliable sources, weak ranking, and incorrect interpretation of retrieved context. Finally, the paper discusses evaluation dimensions such as context relevance, answer faithfulness, and answer relevance, and outlines applications of RAG in education, enterprise search, technical support, research assistance, and document question answering. The analysis concludes that RAG should be viewed as a grounding and evidence-access mechanism rather than a guarantee of hallucination-free generation