The results show that hybrid ensemble methods can improve token-level accuracy in low-resource POS tagging, while also revealing a trade-off between frequent-tag accuracy and rare-tag robustness.
Abstract
Part-of-Speech (POS) tagging is a foundational task in Natural Language Processing (NLP), yet it remains challenging for low-resource and morphologically rich languages such as Amazigh. This paper proposes a hybrid ensemble framework for Amazigh POS tagging that integrates three complementary models: a Bidirectional Long Short-Term Memory network (BiLSTM), a Conditional Random Field model (CRF), and a rule-based morphological analyzer (RBMA). Rather than treating prior results obtained on different corpora and tag inventories as directly comparable, the study evaluates all proposed components under a common 54-tag experimental setting based on the publicly available Amazigh Linguistic Dataset. Three ensemble strategies are examined: majority voting, validation-weighted voting, and logistic-regression stacking. An additional late-fusion ablation applies hard and soft RBMA constraints to CRF and Stacking outputs; hard masking degrades performance substantially, whereas soft masking is more robust but remains below unconstrained decoding. The best micro-level performance is obtained by the stacking ensemble, which reaches 98.51% Micro-F1/accuracy, whereas the boosting-like weighted ensemble obtains the strongest Macro-F1 among the ensemble variants, reaching 74.24%. These results show that hybrid ensemble methods can improve token-level accuracy in low-resource POS tagging, while also revealing a trade-off between frequent-tag accuracy and rare-tag robustness. The findings highlight the usefulness of combining neural, probabilistic, and rule-based information for Amazigh POS tagging, and point to class-balanced meta-learning and character/subword representations as important directions for improving rare and out-of-vocabulary categories.
Alation studies reveal a strong synergy between POS features and CRF decoding, with the combined approach yielding the best single-model performance, and ensembling models trained with different objectives improves both overall F1 score and discontinuous MWE scores, demonstrating the importance of training diversity for capturing non-adjacent syntactic patterns.
Yunus Karatepe, Mert Sülük, Zeynep Tu˘gçe Kırımlı et al.· 0 citations
Part-of-speech (POS) tagging for low-resource languages is limited more by the expense of generating labeled training data than by algorithmic constraints. This study evaluates the extent to which a length-stratified, minimally-supervised pipeline can recover tagging accuracy for the Assamese language using a labeled seed of only 3 to 5 sentences per distinct sentence length (142 to 228 sentences in total), compared to a weak-supervision classifier trained on a conventional 80% corpus split (2,840 sentences). An evidence-gated cascade is constructed, comprising Brown-style word clustering, a seed-ambiguity audit with context-based disambiguation, confidence-gated cluster labeling, and empirically validated rule fallbacks. This approach achieves 94.9% to 95.0% accuracy but only 19.8% to 20.0% token coverage from its minimal seed. To address the coverage gap, every remaining token is backfilled using a classifier trained on the pipeline’s accumulated evidence, resulting in 100% coverage at 84.7% to 86.0% accuracy (depending on seed size), as evaluated against a fixed, shared test set for direct comparison with the larger-budget baseline (90.8% accuracy, same coverage, same test set). Three negative results are reported alongside the positive findings: Brown-style clustering provides negligible benefit at this scale, Viterbi sequence decoding does not transfer to the backfill classifier from the larger-budget setting, and naive self-training fails due to confirmation bias. Additionally, through five replicated random seed draws per condition, it is demonstrated that the residual seed-size effect (3 versus 5 sentences per length, closing approximately 21% of the accuracy gap to the larger-budget baseline) is statistically significant rather than the result of a single favorable draw.
B. Sarma, Rupam Baruah, Diganta Baishya· International journal of com...· 0 citations
One of the basic Natural Language Processing (NLP) tasks is Part-of-Speech (POS) tagging, which helps in various applications like sentiment analysis and information retrieval. However, creating accurate POS taggers for low-resource African languages continues to be difficult due to the scarcity of linguistic resources that are annotated. Its contribution is a deep learning method for POS tagging of Dholuo, a less-resourced Western Nilotic language, spoken by about four million people in Kenya and Tanzania. The suggested system uses DistilBERT, a small transformer model, in addition to FastText and Word2Vec vector representations of words that are used to capture the context and meaning of a word. The KenCorpus Dholuo POS dataset was carefully preprocessed, normalized, and standardized with the Universal POS tags and balanced using a hybrid resampling strategy to bring about class representation. The proposed method combines contextual transformer representations with complementary word representations and a training strategy that is optimized for the linguistic features of Dholuo, while previous studies primarily used multilingual transformer models or traditional sequence-labeling methods. The framework developed is a computationally efficient one that is well-suited for low-resource language processing. The results of the experiments reveal that the proposed DistilBERT-based model outperforms the baseline Conditional Random Field (CRF) and Bidirectional Long Short-Term Memory (BiLSTM) models with an accuracy 79.05%, precision 80.63%, recall 79.05% and F1-score 79.24%. To our best knowledge, these results are the best reported for Dholuo POS tagging, and for under-resourced languages in Africa in general, highlighting the suitability of lightweight transformer architectures.
Maureen Otieno, L. Wanzare, Calvins Otieno· International Journal of Com...· 0 citations
AssistEM, a framework for efficient LLM adaptation to EM via principled data selection, demonstrates that selective fine-tuning not only accelerates adaptation but also improves training efficiency (requiring fewer GPU hours), enabling open-source LLMs to rival–and in some cases outperform–closed-source models.
John Bosco Mugeni, Steven J. Lynden, Toshiyuki Amagasa et al.· International Journal of Dat...· 0 citations
Abstractive text summarization remains a fundamental challenge in Natural Language Processing (NLP), particularly for long documents that require models to preserve long-range dependencies and maintain semantic coherence. Although Transformer-based architectures have achieved strong summarization performance, their full self-attention mechanism scales quadratically with sequence length and often requires input truncation in long-context applications.
This study presents a Neural Turing Machine (NTM)-based framework for abstractive text summarization. The proposed architecture combines a two-layer Bidirectional Long Short-Term Memory (BiLSTM) controller with an addressable external memory bank. Differentiable read and write operations decouple contextual storage from recurrent computation, enabling the persistent retrieval of salient information across extended input sequences. Detailed preprocessing, implementation, training, decoding, and evaluation settings are provided to support reproducibility. The framework was evaluated on the CNN/Daily Mail benchmark and compared with LSTM, Transformer, and BART baselines, as well as published high-performing systems, including PEGASUS, SimCLS, and BRIO. Ablation studies, learning-rate sensitivity analysis, long-context evaluation, inference-time scaling, statistical testing, qualitative error analysis, and memory-access visualization were also conducted.
The proposed NTM model achieved ROUGE-1, ROUGE-2, ROUGE-L, and BLEU scores of 47.8, 23.5, 44.6, and 20.1, respectively. Under the controlled experimental protocol, it outperformed the evaluated LSTM, Transformer, and BART baselines. Comparisons with published results indicate that the model is competitive with recent high-performing summarization systems. The additional analyses demonstrate that the external memory mechanism improves contextual retention and summarization stability, particularly for longer input sequences, while exhibiting favorable inference-time scaling behavior.
These findings demonstrate that integrating an addressable external memory with a BiLSTM controller offers an effective approach to abstractive summarization, particularly when processing long documents. The proposed framework provides competitive summarization performance while reducing dependence on computationally expensive full self-attention. The results highlight the potential of external-memory architectures as a scalable and stable alternative for long-context text summarization.
K. Katti, Kartikeya Reddy Katti, Amanul Islam· Frontiers in Artificial Inte...· 0 citations
Named entity recognition (NER) in low-resource, morphologically rich languages such as Assamese (ISO 639-3: asm) remains a significant challenge due to the scarcity of annotated corpora and the limited applicability of models designed for resource-rich languages. Existing Assamese NER resources suffer from critical limitations: WikiAnn provides broad language coverage but insufficient data volume for neural model training; AsNER, while a gold-standard corpus, supports only five entity categories and lacks a formal tagging scheme, restricting its utility for downstream tasks such as relation extraction and information retrieval. Furthermore, prior Assamese NER systems have relied predominantly on traditional tagging approaches and classical machine learning methods, with limited exploration of modern pre-trained language models and linguistically motivated post-processing strategies. To address these gaps, we present NerAxom, a BIO-tagged NER dataset for Assamese comprising 4,173 sentences and 106,046 tokens annotated across seven entity categories: Person (PER), Location (LOC), Organization (ORG), Date (DATE), Work_of_Art (WOA), Occupation (OCC), and Number (NUM). The dataset was independently annotated by two trained native speakers, achieving an inter-annotator agreement of κ=0.82 (Cohen’s Kappa), with disagreements resolved through expert linguist adjudication. We evaluate NerAxom using two modeling paradigms: (i) a BiLSTM–CRF model with an attention mechanism, tested with FastText, BERT, and MuRIL embeddings; and (ii) direct fine-tuning of the MuRIL transformer as a token classifier. Among embedding-based models, MuRIL yields the highest F1-score of 68%, outperforming FastText (62%) and BERT (64%). Fine-tuning MuRIL directly as a token classifier achieves an F1-score of 70%, establishing a competitive transformer baseline. To address entity misclassifications arising from Assamese morphological complexity, we further introduce a set of language-specific post-processing rules based on morphological suffixes and keyword cues. The hybrid system combining MuRIL embeddings in the BiLSTM–CRF+Attention architecture with these linguistic rules achieves an F1-score of 71% and an overall accuracy of 82% on Assamese Wikipedia biographical text, competitive with the fine-tuned MuRIL transformer (70% F1) and demonstrating that linguistically informed post-processing provides complementary gains over embedding-based neural baselines. The rule component yields the largest category-wise gains for LOC (+20 F1), WOA (+11 F1), and ORG (+9 F1). The NerAxom dataset is publicly available to support further NER research in Assamese and related low-resource Indic languages.
Punam Sarmah, M. Lahkar, Shobhanjana Kalita et al.· Engineering Research Express· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
