Development of an Information Service Chatbot for University Websites Based on Natural Language Processing (NLP) and Retrieval-Augmented Generation (RAG)
TL;DR
It is concluded that an NLP- and RAG-based chatbot is feasible to be implemented as a digital campus information service and improves answer accuracy from 58.5% to 89.25%, with an average response time below three seconds.
Abstract
Academic information services on university websites still largely rely on static systems and complex navigation menus, making it difficult for students and prospective students to obtain information quickly. This study aims to develop an information service chatbot for the Universitas Serambi Mekkah website that integrates Natural Language Processing (NLP) and Retrieval-Augmented Generation (RAG) in an end-to-end manner. The Research and Development (R&D) method was applied, covering system design, implementation, and testing. The system was built using a client-server architecture with React.js on the frontend and FastAPI on the backend, the text-embedding-3-small model from OpenAI for vector representation, ChromaDB as the vector database, and a Large Language Model accessed through OpenRouter for answer generation. Official university documents were extracted using PyPDF as the knowledge base. The testing results indicate that integrating NLP and RAG improves answer accuracy from 58.5% (baseline LLM without RAG) to 89.25%, with an average response time below three seconds. User satisfaction testing obtained an average score of 4.22 out of 5 (very good category). This study concludes that an NLP- and RAG-based chatbot is feasible to be implemented as a digital campus information service. Note: the quantitative results reported here are illustrative and must be replaced with actual field measurements prior to final publication.