2026· Islamic University Journal of Applied Sciences· Vol VIII, pp. 39-56· 0 citations
TL;DR
The results show that BART achieves the best performance with an ROUGE-2 F1-score of 0.40664, while T5 demonstrates superior grammatical acceptability, achieving 93.36%, but BART achieves a very near performance to T5.
Abstract
The rapid growth of academic literature has intensified the need for effective automatic text summarization techniques capable of producing concise and informative representations of scientific documents. While extractive methods are widely used, they are limited in their ability to generate coherent and semantically rich summaries. Recent advances in Transformer-based architectures have enabled significant progress in abstractive summarization; however, their effectiveness on domain-specific datasets, such as scientific articles, remains an open challenge. In this study, we investigate the performance of three pre-trained Transformer-based models—T5, BART, and GPT-2—on the task of abstractive summarization using the CL-SciSumm 2019 dataset. A total of 19 experimental configurations are conducted to analyze the impact of generation parameters, including beam size, length penalties, and n-gram constraints, on summarization quality. The models are evaluated using ROUGE metrics, with a focus on ROUGE-2.To complement content-based evaluation, this work incorporates linguistic acceptability assessment using the Corpus of Linguistic Acceptability (CoLA), a benchmark dataset for evaluating grammatical correctness. The results show that BART achieves the best performance with an ROUGE-2 F1-score of 0.40664, while T5 demonstrates superior grammatical acceptability, achieving 93.36%, but BART achieves a very near performance to T5. Ultimately, these findings demonstrate the potential of pre-trained neural networks, particularly the BART architecture, to drive the future of complex, generative NLP applications, transforming how academic research is processed and understood.
The increasing number of scientific publications has made literature screening more time-consuming, particularly for researchers who need to identify the main contribution of an article before reading the full text. This study develops an extractive summarization model for Indonesian scientific articles using a BERT-based transfer learning approach. The proposed method represents each sentence with contextual embeddings and selects relevant sentences based on their similarity to the document representation, while applying a redundancy threshold to reduce redundancy. A curated corpus of Indonesian research articles was used for model development and evaluation. The generated summaries were evaluated using ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum, with the article abstract used as the reference summary. The experimental results show that the proposed model achieved a ROUGE-1 score of approximately 33%, indicating it retained important keywords and central information from the source documents. However, the lower ROUGE-2 score suggests that the model still has limitations in preserving phrase-level continuity and sentence coherence. Qualitative analysis also shows that the model can capture the main ideas of scientific articles, although some methodological details and contextual information are occasionally omitted. These findings indicate that BERT-based extractive summarization can support preliminary literature screening, but further improvement is needed through stronger baseline comparison, human evaluation, and redundancy-aware optimization.
Made Hanindia Prami Swari, Puji Lestari Tarigan, Gusti Eka Yuliastuti et al.· JOIV: International Journal...· 0 citations
Fine-tuning abstractive summarization models requires high-quality annotated data. However, obtaining such corpora is expensive and time-consuming, as it requires human annotators to read and comprehend long documents to create accurate summaries. Active learning mitigates this issue by selecting only the most informative instances for annotation, allowing models to achieve competitive results with significantly fewer labels. However, the application of active learning to summarization remains under-explored, and existing studies often suffer from instability and significant computational bottlenecks. To overcome these challenges, we propose LOBSTER (LOss-BaSed acTivE leaRning), a novel active learning framework designed specifically for abstractive summarization. LOBSTER improves performance by prioritizing unlabeled instances semantically similar to the model's current high-loss training examples, enabling the model to explicitly correct its specific weaknesses. Our empirical evaluation across three benchmark datasets and two summarization backbone models demonstrates that LOBSTER consistently matches or outperforms current state-of-the-art approaches while achieving a query selection speedup of up to 665x.
M. Ioannou, Tatiana Passali, George Michalopoulos et al.· 0 citations
This study proposes a novel two-level diagnostic protocol for benchmarking LLM-summarizers based on the stability of the generated summaries and motivates further research towards development of robust, reliable and trustworthy LLM-summarizers.
AI-based text summarization plays a critical role in managing the rapidly increasing volume of textual information by enabling efficient information retrieval and concise knowledge representation. Conventional summarization approaches often rely on heuristic rules and surface-level features, limiting their ability to capture contextual relevance and semantic coherence in long documents. To overcome these limitations, this study develops a deep learning–based extractive text summarization framework using a hybrid Bidirectional Encoder Representations from Transformers (BERT) combined with Bidirectional Long Short-Term Memory (BiLSTM) architecture. The model leverages BERT’s bidirectional transformer encoder to generate rich contextual sentence embeddings and employs a BiLSTM network to model inter-sentence dependencies and document-level structure, enabling effective identification of summary-worthy sentences while preserving original semantic content. The framework is evaluated using the BBC News Summary dataset, which contains professionally written news articles and corresponding human reference summaries. Performance is assessed through sentence-level classification metrics and ROUGE-based evaluation. Experimental results demonstrate strong effectiveness, achieving an overall accuracy of 98.5%, with balanced precision (98.7%), recall (98.2%), and F1-score (98.45%), indicating reliable sentence importance classification. In addition, the model attains competitive ROUGE ((Recall-Oriented Understudy for Gisting Evaluation) scores (ROUGE-1 = 0.56, ROUGE-2 = 0.33, and ROUGE-L = 0.52), confirming substantial overlap with human summaries. These results highlight the robustness, generalization capability, and practical applicability of the suggested BERT–BiLSTM structure for automatic text summarization and knowledge representation.
Dr. RakeshKumar ER, S. Ramachandran, Ayushi Khandal et al.· 2026 7th International Confe...· 0 citations
The exponential growth of scientific literature has intensified the demand for automated summarization systems capable of producing abstracts that are both linguistically fluent and factually reliable. Existing approaches face a fundamental trade-off: encoder-decoder models such as BART and T5 maintain strong factual grounding but produce rigid, extractive outputs, while decoder-only large language models (LLMs) such as Llama and Gemma generate highly fluent text yet remain susceptible to hallucination. This paper proposes a two-stage Synergistic Hybrid Ensemble framework designed to resolve this dichotomy. In Stage 1, a fine-tuned BART-Large model generates a factually grounded scaffold draft from a structured input representation comprising the document title, key sentences, method highlights, and results summary. In Stage 2, a QLoRA-adapted Llama-3.2-1B model performs coherent rewriting and stylistic polishing by conditioning on both the scaffold draft and the original source document. Experiments conducted on the arXiv Scientific Research Papers Dataset using BERTScore and entailment-based Factual Consistency metrics demonstrate that the proposed ensemble achieves a Factual Consistency metrics demonstrate that the proposed ensemble achieves a Factual Consistency score of 0.9140, substantially outperforming BART-Large (0.2890) and Llama-3.2-1B (0.6630) individually. Although the ensemble incurs a marginal reduction in BERTScore (0.8980) relative to Llama-3.2-1B (0.9555), this trade-off is justified given the critical importance of factual reliability in high-stakes scientific discourse. These findings confirm that anchoring the generative capacity of decoder-only LLMs to verified factual scaffolds effectively mitigates hallucination risk, offering a scalable and reproducible solution for high-fidelity scientific abstract generation.
Geoffrey Antonio Arifin, Andrew Widyanata, Henry Lucky et al.· International Conference on...· 0 citations
This work study large language model (LLM)-based simplification of scientific texts and presents a human-in-the-loop workflow that transforms expert summaries into more accessible versions for non-specialists.
Kyuri Im, Michael Färber· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
