Jul 2026· International Research Journal on Advanced Engineering Hub (IRJAEH)· 0 citations· 19 references
TL;DR
Evaluation on a multi-page technical PDF document shows that the hybrid retrieval and re-ranking stages together raise retrieval precision and reduce irrelevant or unsupported answers compared with retrieval limited to a single method, supporting the use of this approach for reliable, document-grounded question answering.
Abstract
Building question-answering systems that can read a document and answer naturally phrased questions about it is difficult when retrieval is left to either keyword matching or dense vector search alone, since each method has blind spots that surface as missed context, near-miss answers, or content invented by the underlying language model. This paper describes an optimized hybrid Retrieval-Augmented Generation (RAG) pipeline built to reduce these failure modes by combining two complementary retrieval signals: dense semantic similarity computed over a FAISS vector index, and sparse lexical scoring computed with BM25. Candidates returned by both retrievers are merged and passed through a Cross-Encoder re-ranking stage that scores each query-passage pair jointly, pushing the most contextually relevant chunks to the top before they reach the language model. Final answers are produced by Google’s Gemini model under a prompt that restricts it to the supplied context, which keeps the output tied to the source document rather than to whatever the model already “knows.” The pipeline is exposed through a Streamlit application that lets a user upload a PDF and ask questions about it in plain language, returning each answer alongside a confidence estimate and the page it came from. Evaluation on a multi-page technical PDF document shows that the hybrid retrieval and re-ranking stages together raise retrieval precision and reduce irrelevant or unsupported answers compared with retrieval limited to a single method, supporting the use of this approach for reliable, document-grounded question answering.
Results demonstrate a 157% improvement in retrieval quality over a Naive-RAG baseline, with only 50 ms additional la tency, while Qwen2-VL-2B-Instruct achieved results comparable to cloud-based models in BERTScore, validate that open-source optimized SLMs, paired with advanced retrieval strategies, can provide competitive performance for document understanding without relying on cloud-based models.
Due to the surge in digital document creation, intelligent systems that can effectively retrieve accurate information from unstructured PDF content have become a necessity. Current keyword-based search techniques lack semantic meaning, while large language models (LLMs), when utilized individually, produce hallucinated results when they are not contextually aware of document content. In this paper, we introduce a novel real-time question answering system for PDF content using a lightweight approach to Retrieval-Augmented Generation (RAG). Our system combines a HuggingFace sentence transformer model for semantic embeddings with a FAISS vector similarity search for efficient retrieval, leveraging a Groq-hosted LLaMA 3.3 70B model for context-grounded answer generation. The system is built using LangChain for orchestration and Streamlit for web-based deployment, and requires no GPU, no fine-tuning, and no pre-indexed corpus. Evaluation across five complementary metrics—Exact Match, Token-level F1, BLEU, ROUGE, and semantic similarity—demonstrates strong alignment between generated and reference answers, with a peak semantic similarity of 0.8709 and a mean Token F1 of 0.4701, confirming the viability of this approach for real-world document intelligence.
Mukesh Lakshmi Sai Medikonda, Kalva Vishnu Teja, K. Greeshma et al.· International Conference Com...· 0 citations
This study proposes STaR, a novel retriever fine-tuning framework that integrates BM25 similarity graph-based soft labeling with a triplet similarity learning strategy based on Sentence-BERT (SBERT), and introduces a triplet-aware SBERT training architecture that explicitly models relative semantic distances between queries and candidate passages, significantly enhancing retrieval ranking precision and semantic robustness.
Jiali Jiang, Chih-Yung Chang, Youxi Li et al.· Multimedia Systems· 0 citations
Although multi-turn inference remains more expensive than single-call retrieval, VecTree-RAG provides a structure-aware and traceable architecture for scientific literature question answering.
Large language models may suffer from insufficient context use and unsupported generation in question answering tasks that require external knowledge. This study compares the main strategies affecting retrieval and generation performance in retrieval-augmented generation systems within a common experimental setting. In addition to a standard dense retrieval baseline, we evaluate multi-query, hypothetical document, hypothetical question, sparse-dense hybrid retrieval, and chunk compression, and we also propose an integrated method (FUSERAG) that combines these components. Experiments are conducted on a dataset containing 223 documents, 7933 chunks, and 1488 question-answer instances. Results show that the proposed method achieves the highest Mean Reciprocal Rank, nDCG at 5, and recall at 5 at the retrieval level, while the sparse-dense hybrid approach yields the best generation results.
Buğra Şimşek, Korhan Sevinç, Helen Parlar et al.· Signal Processing and Commun...· 0 citations
The Adaptive Multi-Stage Vector Retrieval (AMSVR) framework is proposed, prioritising weighted, drift-resistant composition over uniform fusion, and offers tailored configurations: AMSVR-Scientific (dense + tuned hybrid) peaks at NDCG@10 = 0.7570 on SciFact, while AMSVR-Full (seven stages) targets broader, noisier corpora where Recall@100 matters most.
Samsudeen Alabi Bankole, Yakub Kayode Saheed· NLP & Big Data· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
