An Optimization Framework for Retrieval Augmented Generation in Indonesian Educational Question Answering
TL;DR
A RAG optimization framework for Indonesian-language educational question answering using a Human-Computer Interaction learning corpus as a case study is developed and provides a procedure for selecting retrieval and generation settings for a given corpus.
Abstract
The Retrieval-Augmented Generation (RAG) approach has been widely adopted to produce responses that are more closely aligned with a predefined knowledge context. However, many RAG implementations have not undergone systematic optimization of their retrieval and generation components, resulting in outputs that do not always correspond accurately to the reference context. This study developed a RAG optimization framework for Indonesian-language educational question answering using a Human-Computer Interaction learning corpus as a case study. In the retrieval stage, the study evaluated chunking strategies, multilingual embedding models, and the use of a reranker. Evaluation was conducted using Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (nDCG@K), and Hit@K. In the generation stage, candidate Large Language Models (LLMs) were assessed using RAGAS metrics, namely Context Precision (CP), Context Recall (CR), Faithfulness (F), Answer Relevancy (AR), and Answer Correctness (AC). Experimental results showed that the GTE configuration with fixed-size chunking and a reranker yielded the best retrieval performance, achieving an MRR of 0.9082, nDCG@5 of 0.9215, and Hit@5 of 0.9655. In the generation stage, Gemma 4 E4B exhibited the most balanced answer quality. The resulting framework provides a procedure for selecting retrieval and generation settings for a given corpus.