Leveraging Task-Adaptive Continual Pre-Training to Enhance the Classification Ability of Language Models
This study presents a systematic empirical investigation of task-adaptive continual pre-training (TAPT), introduced by Gururangan et al., for Turkish language understanding, with a particular focus on the effect of the masked-language-modeling rate.