Rag Based Intelligent Information Retrieval System for Domain Specific Knowledge Queries

Abstract

In essence, traditional Retrieval-Augmented Generative (RAG) models are only able to process textual data in a unimodal manner making them incapable of being used in practical settings in which vital pieces of information reside within the form of images, tables, graphs, and other forms of scanned documents. In this paper, we propose Full Stack Multimodal Retrieval-Augmented Generation (MM-RAG) that utilizes multimodal document ingestion using the unstructured library with hi_res layout detection, semantic chunking up to a maximum of 2,000 characters per chunk, cross-modal embedding generation using nomic-embed-text with generation of 768-dimensional vectors, hybrid retrieval using the combination of BM25 and kNN cosine similarity using a common OpenSearch 2.11 index, and response synthesis using Gemini-2.0-Flash. The model uses the Gemini-2.0-Flash vision-language model for the generation of semantic descriptions of figures and tables allowing visual content to be entirely searchable. Experiments conducted on the dataset of 50 documents and 200 test queries show that the proposed system provides Faithfulness and Answer Relevancy scores of 0.80 which are 128.6% and 29.0% better than vanilla LLMs and text-only RAG models respectively. Hybrid retrieval provides a Context Precision score of 0.80 which is 31.1% and $\mathbf{1 2. 7 \%}$ superior to keyword-only and semantic-only search. The framework maintains stable scalability with query latency of 1.2 seconds at 100 documents on commodity hardware.