Aug 2026· Revista Brasileira de Computação Aplicada· Vol 18, pp. 1-15· 0 citations
TL;DR
This article presents a procedure to deal with and treat unknown languages or languages without available resources such as Kicongo, a Bantu matrix language, commonly spoken in the northern regions of Angola.
Abstract
Pre-trained contextual language models have demonstrated excellent performance in training with data from new languages and tasks. For this purpose, additional pre-training is necessary, since the lack of vocabulary of the language always tends to degrade the results. In this article, we present a procedure to deal with and treat unknown languages or languages without available resources such as Kicongo, a Bantu matrix language, commonly spoken in the northern regions of Angola, with greater incidence in rural regions and in the countries of central Africa (Democratic Republic of Congo, Republic of Congo and Gabon) and throughout the world. With the extension of Natural Language Processing (NLP) models based on the Bidirectional Encoder Representations from Transformers (BERT) architecture (BERT, RoBERTa and DistilBERT), the performance evaluation of the models was carried out using metrics such as accuracy, precision, recall and f1-score, achieving a performance rate higher than 97%.
Pretrained language models (PLMs) have established state-of-the-art performance across diverse natural language understanding (NLU) tasks. This study reveals that seman-tic-rich explanations of lexical units can effectively guide PLM learning processes. We propose a novel language understanding enhancement method with token interpretation (LUETI) that addresses two critical limitations in conventional PLMs: Incomplete token semantics caused by isolated contextual learning and insufficient semantic encoding in embedding matrices. LUETI operates through dual mechanisms, augmenting token represen-tations by integrating hidden states with corresponding token interpretations and refining embedding spaces using interpretation-derived semantic vectors for token prediction. LUETI, which is implemented as a plug-in module for standard architectures, demonstrates significant improvements on BERT and GLM, achieving average performance gains of 3.36% and 4.87% respectively on the SuperGLUE benchmark with equivalent parameters and training data. Note that LUETI-equipped models attain comparable performance to baseline PLMs using only 60% of pretraining data. Findings establish token interpretation as a computationally efficient but semantically powerful enhancement strategy for language model pretraining.
Tianyi Chen, Yashen Wang, Huan Chang et al.· IEEE/CAA Journal of Automati...· 0 citations
MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish, is presented, providing both a foundation for Yiddish NLP and a practical template for language model development in historically rich but digitally underrepresented languages.
Uri Katz, Omer Goldman, Tomasz Limisiewicz et al.· 0 citations
Today, Large Language Models (LLMs) perform many tasks in the field of natural language processing with high success, from text generation to translation, semantic analysis to code writing. However, these models have some fundamental limitations that make their reliable use challenging. They can produce factual errors known in the literature as hallucinations and cannot directly access developments after their training period. They also sometimes reproduce biases present in the training data. The Re-trieval-Augmented Generation (RAG) approach aims to produce more up-to-date and verifiable outputs by dynamically feeding the model with external information sources, thereby reducing hallucination and temporal limitations. This study examines distributed learning approaches aimed at protecting data privacy and QLoRA-based fine-tuning strategies within a mathematical framework. It also addresses 4-bit NormalFloat (NF4) quantization techniques used to improve system efficiency. Furthermore, it de-tails morphology-aware hybrid access methods and adaptive routing mechanisms that can make deci-sions based on query complexity to achieve results more suitable for Turkish. The study demonstrates that combining RAG-based systems with federated learning and homomorphic encryption creates a secure, decentralized architecture, and that LLM can be efficiently run on low-resource systems using the NF4-QLoRA combination. However, limitations such as encryption latency, erroneous information from untrusted sources, and the inadequacy of standard evaluation methods for Turkish also exist.
M. Toy, Ahmet Ali Süzen· International journal of 3d...· 0 citations
