Code Generation Beyond Mainstream Languages

Abstract

Large Language Models (LLMs) have transformed software engineering through automated code generation, yet their performance varies widely across target languages, specifically between mainstream (e.g., Python) and niche languages (e.g., OCL). Key challenges are determining which LLMs and prompting techniques can most effectively enhance generated code quality and syntactic validity, particularly for low-resource languages. To this end, we propose a framework designed to systematically tackle these two issues. First, diagnosing the LLM’s ability to generate well-formed code in a target language, properly using the language’s keywords and syntactic rules. Second, selecting the best possible configuration of the code generation process that optimizes the quality of LLM-generated code. These findings are particularly relevant to a wide variety of textual languages and notations used in model-driven engineering (MDE).