A Comparative Benchmark of Specialized Deep Learning Architectures and Fine-Tuned LLMs for Arabic Text Readability

Abstract

Readability assessment for Arabic remains challenging due to the language's complex morphology. This paper presents a comparative study benchmarking traditional Machine Learning (ML), advanced Deep Learning (DL), and finetuned Large Language Models (LLMs). Utilizing a dataset of 4,519 Arabic sentences categorized into three proficiency levels, we evaluate models across accuracy and computational efficiency. Our results demonstrate that a hybrid CNN-BiLSTM architecture utilizing AraVec (Word2Vec) embeddings achieves a peak accuracy of 96.68%, outperforming fine-tuned LLMs like Llama3.2-1B (94.69%). We provide empirical evidence of the prohibitive resource demands in LLMs, which required significantly higher training times (14,697s) compared to specialized DL models (162.98). These findings suggest that for discrete Arabic text classification, tailored DL architectures provide a superior balance of precision and resource efficiency.