Jul 2026· Nigerian Journal of Technology· Vol 45, pp. 401-410· 0 citations· 18 references
TL;DR
Results show that fine-tuning and integrating the BAAI/bge-large model significantly improves the chatbot's ability to retrieve relevant information, demonstrating a significant boost in retrieving the most relevant scholarly information for user queries.
Abstract
Foundation models, particularly Large Language Models (LLMs), show promise for chatbots for scholarly research chatbots but suffer from “hallucinations” and limited domain knowledge. This work addresses these reliability issues by developing a Retrieval-Augmented Generation (RAG) chatbot designed explicitly for scholarly research interactions within the Natural Language Processing (NLP) domain. The system utilizes a domain-specific corpus from the Association of Computational Linguistics (ACL) Anthology. The corpus was processed into text chunks and indexed using a high-dimensional vector database to facilitate efficient information retrieval. The generation component comprises an instruct-tuned LLM; Mixtral 8x7b, which synthesizes the retrieved context into coherent, evidence-based responses while maintaining the original semantic integrity of the scholarly source. Retrieval is performed using semantic similarity search. To improve dense retrieval precision, the BAAI/bge-large Sentence Transformer model was fine-tuned on a synthetic dataset generated from the corpus. Results show that fine-tuning and integrating the BAAI/bge-large model significantly improves the chatbot's ability to retrieve relevant information. Compared to the base model, retrieval accuracy of the fine-tuned model increases by up to 15% across different metrics: Accuracy@k, Precision@k, Recall@k, where k represents the count of retrieved chunks considered for evaluation, and Mean Reciprocal Rank (MRR@10). Notably, accuracy reaches an impressive 97%, demonstrating a significant boost in retrieving the most relevant scholarly information for user queries. These findings underscore the efficacy of fine-tuned Retrieval-Augmented Generation (RAG) systems in developing reliable, grounded chatbots for academic environments.
The results show that the RAG architecture provides a scalable alternative for creating precise, contextually grounded conversational agents, thereby mitigating some of the main drawbacks of LLMs.
Rabia Shabbir, K. Talpur, Shakeel Ahmad· ICCK Transactions on Machine...· 0 citations
A Retrieval-Augmented Generation (RAG)-based chatbot for academic regulation information services using a locally deployed Large Language Model (LLM) and promising feasibility for supporting document-based academic regulation information services is indicated.
This paper presents the design and implementation of intelligent chatbots using Retrieval-Augmented Generation architecture to derive accurate, context-based responses from domain-specific knowledge sources such as PDFs and CSV files. The proposed system integrates information retrieval mechanisms with large language models to overcome crucial limitations of traditional chatbots: hallucination, knowledge cut-off, and inability to access private organizational data. The proposed architecture uses a six-stage pipeline consisting of document ingestion, text splitting, and embedding generation with sentence transformers, semantic retrieval, and response generation via Large Language Models (LLMs). By preprocessing the PDF and CSV documents using a RecursiveCharacterTextSplitter and converting them into 384-dimensional semantic embeddings, the system works via a similarity-based retrieval of document chunks. For implementation, the LlamaIndex framework was used for data indexing and query orchestration, and HuggingFace Transformers for language modeling. Experimental validation conducted on the smart campus chatbot system demonstrated significant improvements in the accuracy, relevance and transparency of responses across education, student services, and administrative support domains compared with a standalone language model. Because of its modular design, it supports a wide range of domain-specific topics without the combinatorial costs of re-training. This is another important application for systems that require regular updates and accurate information. Overall, the proposed framework provides a simple and practical approach for easily implementing domain-aware chatbots.
Varsha Spoorthi Shree Narravula, V. K, Yashaswini B. Suresh et al.· 2026 7th International Confe...· 0 citations
Universities now use Large Language Models (LLMs) to transform their processes for managing student information. The paper introduces an upgraded chatbot system for Narasaraopeta Engineering College (NEC) which extends previous on-premise LLM chatbot research by providing four new functions. The system uses (1) Retrieval-Augmented Generation (RAG) to create citation-based responses through LlamaIndex and ChromaDB, (2) Context Memory which maintains conversation flow during multiple dialogue exchanges, (3) Voice Input through OpenAI Whisper Speech-to-Text (STT) technology, and (4) Multilingual Support which covers English and these seven languages: Hindi, Telugu, Tamil, Kannada, and Malayalam through IndicNLP. The system tested 60 benchmark questions across four academic categories which included regulations and examination policies and fee structures and multilingual queries and achieved 96.7% overall accuracy with sub-second text response times and 1.0–1.4 second voice response times. The system operates entirely on-premise through Docker which safeguards institutional data privacy while eliminating the need for recurring cloud API expenses. The upcoming development will create Emotion-Aware AI, FAQ Auto-Learning, Student Portal Integration, and a Mobile Application.
M. Yaswanth, Kopparapu Sai Amar Durgesh, Mogili Harsha Vardhan et al.· 2026 7th International Confe...· 0 citations
Results suggest that code-mixing-aware RAG chatbots can meaningfully improve DSA learning outcomes while remaining feasible for instructor-led deployment in low-resource settings.
Saroj Bhandari, Puja Dhital· Aadim Journal of Multidiscip...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.