DocuMind: An Offline Multilingual Retrieval-Augmented Generation System for Document Question Answering Using Local Large Language Models

Abstract

ABSTRACT DocuMind is a fully offline, privacy-preserving, multilingual Document Question Answering system built on the Retrieval-Augmented Generation (RAG) architecture. The system enables users to upload documents in any language and ask questions in their native language — including Kannada, Hindi, Tamil, Telugu, and English — receiving accurate, contextually grounded answers from a locally running Large Language Model (LLM) with zero cloud dependency and zero recurring cost. The system employs Language-Agnostic BERT Sentence Embeddings (LaBSE), a multilingual model supporting 109 languages, to map all text into a shared 768-dimensional semantic vector space, enabling true cross-lingual retrieval without any translation step. Document embeddings are persisted in a local ChromaDB vector database and answer generation is performed by LLaMA 3.2 (3B parameters) running locally via the Ollama runtime. Evaluation using RAGAS-style metrics across five languages demonstrates Faithfulness scores of 0.77 to 0.87 and average query latency under 8 seconds on a standard consumer laptop with 8 GB RAM. KEYWORDS Retrieval-Augmented Generation, Large Language Models, Multilingual NLP, LaBSE, ChromaDB, Ollama, LLaMA, Document Question Answering, Indic Languages, Offline AI, Natural Language Processing, Cross-lingual Retrieval.