A Systematic Review of Transfer Learning and Data Augmentation in Neural Machine Translation of Low-Resource Languages
TL;DR
The findings show that a hybrid combination of TL and DA with a self-supervised objective is the most effective solution for extremely low-resource scenarios, capable of producing the highest translation quality and outperforming baseline models and traditional methods such as Statistical Machine Translation (SMT).
Abstract
Modern Neural Machine Translation (NMT) systems have achieved state-of-the-art, performance, largely due to the availability of large-scale parallel corpora. However, the translation quality of NMT for Low-Resource Languages (LRL) remains limited due to data sparsity. Numerous studies have proposed different strategies to address this challenge. Among the most widely adopted strategies are Transfer Learning (TL) and Data Augmentation (DA) strategies. This research aims to present a systematic review of how these techniques, including Back-Translation (BT), Hybrid Transfer Learning (HTL), and the utilization of self-supervised objectives such as Masked Language Modeling (MLM), Causal Language Modeling (CLM), and Denoising Autoencoder (DAE), affect the quality improvement of NMT for LRL. The findings show that a hybrid combination of TL and DA with a self-supervised objective is the most effective solution for extremely low-resource scenarios, capable of producing the highest translation quality (highest BLEU score) and outperforming baseline models and traditional methods such as Statistical Machine Translation (SMT).