Progress in natural language processing (NLP) has primarily benefited high-resource languages. Although Arabic now has access to substantial corpora and widely used pretrained language models such as AraBERT and MARBERT, it still exhibits a significant imbalance, particularly in the availability of structured lexical–s...
Leila Nouri, H. Aliane· ACM Transactions on Asian an...· 0 citations
A bidirectional Mamba encoder pretrained via masked language modeling on a corpus combining Arabic Wikipedia and CulturaX text is introduced, trained end-to-end on four consumer-grade NVIDIA RTX 2080Ti GPUs (11GB) over approximately ten days.
This paper proposes a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models, and reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance.
A. A. Aliane, N. Semmar, H. Aliane· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.