Towards Responsible AI in Low-Resource Languages: Evaluating LLM Responses with a Culturally Aware Sinhala Dataset

Abstract

Large Language Models (LLMs) are increasingly used as primary information sources, raising critical concerns about their safety and reliability across languages. While prior work has established the importance of harmless yet non-evasive responses, these guarantees are largely evaluated in high-resource languages. In this paper, we investigate the safety of responses in LLMs in Sinhala, a low-resource language with strong cultural grounding. We introduce a framework that extends existing definitions of harmlessness by incorporating responsible behaviour, requiring models to actively discourage harmful actions and provide safety-oriented guidance for culturally grounded question prompts. Using a curated set of culturally grounded and safety-critical queries, we evaluate multiple publicly available LLMs across categories, including self-harm, medical advice, culturally sensitive topics, and illegal activities. Our findings reveal significant safety gaps, including the generation of dangerous misinformation and the absence of appropriate intervention in high-risk scenarios. We further propose a Sinhala-specific diagnostic dataset to evaluate the safety of responses generated by LLMs in the Sinhala language.