Detection of AI-Generated Khasi Text Using Machine Learning Techniques

Abstract

The widespread adoption of large language models (LLMs) has made AI-generated text increasingly difficult to distinguish from human writing, even in languages previously considered beyond the reach of generative AI. For Khasi, an Austroasiatic language spoken in Meghalaya, India, this presents a pressing problem: no prior work exists on detecting AI-generated Khasi text, no labeled classification dataset is publicly available, and the NLP toolchain for Khasi remains extremely thin. This paper addresses that gap directly. We compiled the first labeled corpus for this task, containing ~56,000 sentences (~28,000 human and ~28,000 AI generated text), sourced from established Khasi news websites and a synthetic parallel corpus. Twelve machine learning models were evaluated, spanning traditional classifiers (Logistic Regression, SVM, FastText), recurrent architectures (LSTM, BiLSTM, GRU, and biGRU), and transformer-based models (mBERT, DistilBERT Multilingual Cased, mmBERT, XLM-RoBERTa, and KhasiBERT). On the sentence-level test set, biGRU achieved the best F1-score of 0.7832. Out-of-distribution evaluation on 309 unseen document-level texts revealed, however, that calibration is critical: mmBERT demonstrated the most reliable uncertainty modeling while KhasiBERT exhibited severe over-confidence. These findings establish the first systematic benchmark for AI-generated text detection in Khasi, providing concrete insights for future work in low-resource language NLP.