Bridging the Language Gap in Text-to-SQL: Adapting LLMs for Chichewa in a Low-Resource Setting
TL;DR
The adaptation of LLMs for Text-to-SQL generation in Chichewa, a low-resource Bantu language spoken by over 12 million people in Malawi and neighboring regions, is investigated and parameter-efficient fine-tuning (QLoRA) is applied to selected models and the combined effect of QLoRA fine-tuning with retrieval-augmented prompting is evaluated.
Abstract
Recent advances in Large Language Models (LLMs) have significantly improved Text-to-SQL performance in high-resource languages. However, their effectiveness in low-resource language settings remains largely underexplored. In this work, we investigate the adaptation of LLMs for Text-to-SQL generation in Chichewa, a low-resource Bantu language spoken by over 12 million people in Malawi and neighboring regions. We construct a structured Chichewa Text-to-SQL benchmark consisting of 400 manually curated natural language–SQL pairs grounded in a unified relational database covering agriculture, commodity prices, population statistics, market data, and food insecurity. We systematically evaluate five open-source LLMs under zero-shot, random 5-shot, and retrieval-augmented 5-shot prompting, in both English and Chichewa. We then apply parameter-efficient fine-tuning (QLoRA) to selected models and, crucially, evaluate the combined effect of QLoRA fine-tuning with retrieval-augmented prompting. QLoRA alone improves English execution accuracy to 78.3% and Chichewa execution accuracy to 41.7%. When combined with retrieval-augmented prompting, QLoRA achieves 53.3% execution accuracy in Chichewa, representing the best reported result for this language on this benchmark and narrowing the English–Chichewa gap to 23.4 percentage points. Our findings offer practical guidance for deploying database interfaces in linguistically underserved environments.