Large Language Models (LLMs) are increasingly used for translation, yet their value depends on preserving meaning rather than producing fluent output. This study evaluates seven LLMs on Japanese–Croatian translation, a low-resource, typologically distant language pair. Using rubric-based human evaluation of adequacy, fluency, terminology, and register, we compare model performance. Results show a stable ranking: qwen3 performs best, followed by phi4 and gemma3, while qwen2 performs worst. Performance differences reflect structural reconstruction, particularly argument recovery, aspectual mapping, lexical precision, and register. Qualitative analysis also reveals limited differentiation within the South Slavic continuum and pragmatic inconsistencies. Although productivity effects were not measured, improved translation adequacy may reduce post-editing and verification effort.
The evaluation of various NMT and LLM architectures specifically for Croatian from/to English and Spanish demonstrates that open–source models can achieve, and occasionally surpass, the quality of Google Translate, a widely used commercial NMT system.
Antoni Oliver, Sergi Álvarez–Vidal· Suvremena Lingvistika· 1 citation
J-PragEval-v0 is introduced, a minimal-pair benchmark isolating four such phenomena from surface fluency, and Pragmatic Representation Steering is specified, a parameter-free inference-time method that edits residual-stream activations along the class-mean-difference directions probing identifies.
Large language models (LLMs) increasingly evaluate human writing in high-stakes domains such as hiring and academic assessment, putting non-native speakers at particular risk. Drawing on the language attitudes framework, we compared human and LLM evaluations of parallel L1- and L2-written Japanese emails on three dimen...
The results show that LLMs and Google Translate consistently outperform specialized MT systems in terms of fluency, meaning preservation, and lexical-thematic alignment.
Beatriz Ribeiro Borges, P. H. R. Gabriel, E. Faria· International Journal of Dat...· 0 citations
The concept of temperature is introduced as a means for controlling stochasticity in LLM-based translation as well as a practical and replicable experiment conducted in Google Colab to establish a replicable methodological pathway.
Celia Rico Pérez· Digital Translation· 0 citations
The study concludes that terminological consistency should be considered as a separate dimension of translation quality and emphasizes the importance of terminology control and human post-editing in AI-assisted political translation.
Osama Bala· (Faculty of Arts Journal) مج...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
