A Survey of conversational AI from rule based to generative and retrieval augmented generation chatbots

Abstract

Retrieval-Augmented Generation (RAG) has rapidly emerged as a foundational paradigm for knowledge-grounded conversational agents, particularly in high-stakes domains where factual accuracy, transparency, and adaptability are paramount. Although prior surveys have examined the evolution of chatbots or the mechanics of retrieval-augmented language models in isolation, a comprehensive, system-level synthesis of RAG-based chatbots as end-to-end conversational architectures remains limited. This survey addresses that gap by presenting a structured, design-oriented analysis of RAG-driven conversational systems through a principled framework that decomposes architectures along critical dimensions, including document segmentation and chunking strategies, embedding and indexing mechanisms, retriever and re-ranking models, knowledge integration and grounding techniques, attribution mechanisms, and evaluation methodologies. Following a transparent and reproducible literature selection protocol, we examine representative RAG implementations across both open-domain and specialized settings, systematically analyzing architectural trade-offs, interaction effects among components, recurring failure modes, and sources of performance variability that are often obscured by aggregate benchmark metrics. Beyond cataloguing techniques, the survey advances a conceptual understanding of when and why RAG systems succeed or degrade, critiques prevailing evaluation practices, and delineates emerging research frontiers such as retrieval robustness under distributional shift, attribution-aware and verifiable generation, privacy-preserving retrieval pipelines, and multimodal grounding. By reframing the discourse from method enumeration to architectural reasoning and system-level design principles, this work provides a rigorous foundation for researchers and practitioners seeking to develop reliable, interpretable, and scalable RAG-based conversational agents. While rule-based and purely generative chatbots are discussed as essential historical and conceptual context, the primary analytical focus of this survey is on RAG-driven conversational architectures, which represent the current frontier of knowledge-grounded dialogue systems.