Skip to content
Conference

Data-Centric Evaluation of Arabic Abstractive Summarization Using a Large-scale Curated News Corpus

Aug 2026 · 2026 6th International Conference on Emerging Smart Technologies and Applications (eSmarTA) · pp. 1-8 · 0 citations · 17 references

Abstract

This paper introduces MAAD, a high-quality, carefully constructed and curated by the authors large-scale Arabic dataset for abstractive news summarisation. The authors selected a high-quality subset of 50,000 articles from the dataset Original, which contains 602,792 articles. To maintain the quality, diversity, and training suitability of the subset, the subset underwent a multi-stage preprocessing pipeline involving noise removal, duplicate filtering, linguistic normalisation, and expert validation. The experimental evaluation was executed in two phases. In the first phase, three transformer-based models (ArabicT5, AraBART, and mT5) were evaluated on a controlled subset of 1,110 articles to establish fair baseline comparisons among models, where ArabicT5 achieved the best performance (ROUGE-1: 23.64, ROUGE-2: 11.82, ROUGE-L: 22.10). In the second phase, ArabicT5-base was trained on all 50,000 articles to evaluate scalability, achieving substantially improved results of 68.4, 52.3, and 64.1, respectively, with a BLEU score of 58.7. The findings emphasise the significance of scale, effective preprocessing, and the benefits of Arabic-specific pretraining on the quality of summarisation. Moreover, a human evaluation on 500 randomly sampled instances verified fluency and adequacy scores of 4.86 and 4.35, respectively, with a strong inter-annotator agreement (Cohen's Kappa: 0.78 and 0.74). Overall, the findings indicate that MAAD is a reliable and scalable dataset with strong potential to serve as a benchmark for Arabic abstractive summarisation and to support the development of robust transformer-based models.

View source

Similar papers

Open access Jul 2026

Enhancing Readability of Telugu Text Summarization Using Multi-scale Attention and Bio-inspired Optimization

The proposed MLOA-MA-ASeqNet architecture, a Multi-scale Attention and Adaptive Sequence-to-Sequence Network whose hierarchical encoder operates simultaneously at word, phrase and sentence level granularity, achieves the highest average score across fluency, adequacy, coherence and readability.

V. M, Kunal Chakma, Anupam Jamatia et al. · 0 citations
Open access Jul 2026

Implementation of Transfer Learning for Automatic Summarization in Research Article Synthesis

The increasing number of scientific publications has made literature screening more time-consuming, particularly for researchers who need to identify the main contribution of an article before reading the full text. This study develops an extractive summarization model for Indonesian scientific articles using a BERT-based transfer learning approach. The proposed method represents each sentence with contextual embeddings and selects relevant sentences based on their similarity to the document representation, while applying a redundancy threshold to reduce redundancy. A curated corpus of Indonesian research articles was used for model development and evaluation. The generated summaries were evaluated using ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum, with the article abstract used as the reference summary. The experimental results show that the proposed model achieved a ROUGE-1 score of approximately 33%, indicating it retained important keywords and central information from the source documents. However, the lower ROUGE-2 score suggests that the model still has limitations in preserving phrase-level continuity and sentence coherence. Qualitative analysis also shows that the model can capture the main ideas of scientific articles, although some methodological details and contextual information are occasionally omitted. These findings indicate that BERT-based extractive summarization can support preliminary literature screening, but further improvement is needed through stronger baseline comparison, human evaluation, and redundancy-aware optimization.

Made Hanindia Prami Swari, Puji Lestari Tarigan, Gusti Eka Yuliastuti et al. · 0 citations
Open access Aug 2026

ArabicOpinion: A Five-Dimensional Multi-Granular Evaluation Framework for Arabic Opinion Summarization

Progress in Arabic opinion summarization has been limited by the absence of standardized evaluation frameworks, despite Arabic being the fifth most widely spoken language globally. We present ArabicOpinion, the first large-scale, human-evaluated benchmark and automated evaluation framework for Arabic opinion summarization. The framework introduces entity-aware evaluation that assesses named entities alongside opinions, addressing a key gap in existing work. It includes evaluation metrics covering five dimensions: relevance, faithfulness, sentiment preservation, abstractiveness, and genericity, with a bidirectional multi-granular architecture that measures the first two independently. We benchmark GPT-4o, Claude, and JAIS. The automated metrics show strong alignment with human judgments. Opinion-level Matryoshka embeddings perform best for relevance and faithfulness, while AraBERT-Restaurant-Sentiment achieves the highest sentiment preservation accuracy. These results provide a standardized foundation for evaluating Arabic opinion summarization and offer insights into the performance of large language models on Arabic content.

Bayan Aldashnan, Abdulrahman Alothaim, Ahmed Alsanad · 0 citations
2026

Assessing Transformer Models for Abstractive Summarization of Scientific Articles

The results show that BART achieves the best performance with an ROUGE-2 F1-score of 0.40664, while T5 demonstrates superior grammatical acceptability, achieving 93.36%, but BART achieves a very near performance to T5.

Emad Nabil · 0 citations
Open access Jul 2026

LSTM-Based Classification of Indonesian Regional Song Lyrics by Language

This study successfully proposes a Long Short-Term Memory (LSTM)-based model for automatic classification of Indonesian regional song lyrics by language, demonstrating that LSTM effectively captures sequential linguistic patterns and contextual relationships within regional languages.

Muhammad Rizky, Anandita Priatama, Aviv Yuniar Rahman et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.