Domain-Adapted Language Models for English-Turkish Medical Text Translation: An Evaluation of Zero-Shot, Few-Shot, and Fine-Tuning Strategies

Abstract

This study investigates the English-to-Turkish clinical translation performance of three large language models (Aya-23, Seed-X-Instruct, Gemma-3) using zero/few-shot prompting and QLoRA fine-tuning. Our three-stage setup evaluates: (1) general translation after FLORES-devtest training, (2) medical domain adaptation on i2b2 via a mixed dataset, and (3) generalization capacity on the independent FLORES-test set after mixed-data fine-tuning, using SacreBLEU, chrF, and COMET. Results show that Seed-X-Instruct achieves the strongest fine-tuned performance, reaching 32.70 BLEU, 64.34 chrF, and 90.65 COMET on the general FLORES test set, and 22.40 BLEU, 54.61 chrF, and 79.70 COMET on the medical i2b2 set. After mixed-data training, Seed-X improves from 11.48 to 32.39 BLEU on the unseen FLORES test set, whereas Aya-23 drops from 19.23 to 10.83 and Gemma-3 largely preserves its strong zero-shot baseline. These findings indicate that QLoRA effects are model-dependent and that domain adaptation does not yield the same generalization behavior across models.