Natural Language Processing-Driven Chatbot for Algorithm Learning: A Retrieval-Augmented Generation Approach using Romanized Nepali and English

Abstract

Data structures and algorithms (DSA) form a cornerstone of computer science education, yet many students in Nepal struggle with the cognitive load imposed by English-only learning materials when their habitual conceptual language is a code-mixed register of Romanized Nepali and English. This paper presents the design, implementation, and classroom evaluation of an NLP-driven educational chatbot, AlgoSathi, that addresses this gap through a Retrieval-Augmented Generation (RAG) architecture tailored to code-mixed pedagogical queries. The system combines a curated DSA knowledge base of 1,252 chunks drawn from instructor-authored lecture notes, textbook material, and worked solutions; a multilingual sentence encoder (paraphrase-multilingual-mpnet-base-v2) backed by an HNSW-indexed FAISS vector store; a hybrid BM25-plus-dense retriever with a cross-encoder re-ranker; and a large language model generator constrained by a pedagogically structured prompt that preserves the language register of the learner's query. A 12-week quasi-experimental study with 52 undergraduates (26 experimental, 26 control) showed the chatbot group achieved a 26.2-point mean learning gain versus 14.5 points for controls (t (50) =4.21, p<0.001, d=1.17, 95% CI for gain difference: 6.2-17.2pp). ANCOVA using pre-test scores as a covariate confirmed a significant adjusted post-test advantage of 12.1 points (95% CI: 7.4–16.8, p < 0.001). Statistical assumption checks (Shapiro-Wilk, Levene's test) and a priori power analysis (power = 0.97) are reported. The retrieval  pipeline reached P@5=0.79 on code-mixed queries, with student satisfaction averaging 4.31/5. Results suggest that code-mixing-aware RAG chatbots can meaningfully improve DSA learning outcomes while remaining feasible for instructor-led deployment in low-resource settings.