General-purpose named entity recognition using transformer-based fine-tuned approach

Abstract

Named Entity Recognition (NER) is a fundamental component of Natural Language Processing. It enables structured understanding of textual data across various domains such as news articles, Wikipedia, government records, social media, and other general data. NER is also useful for down-stream tasks like text summarization (summarizing text by identifying key entities and events), text classification, information retrieval and filtering, question-answering, and many more. The research article proposes a transformer-based fully fine-tuned XLNet model with 117M parameters. The model is designed to recognize and extract named entities form general domain data. A full fine-tuning approach is employed, updating all model parameters. Differential optimization rules are applied to distinct parameter groups, with normalization layer parameters (bias, gamma, beta) excluded from weight decay. This strategy helps to enhance model generalization and decrease the risk of overfitting. A full fine-tuning was employed on transformer-based models such as Bidirectional Encoder Representations from Transformers (BERT), DistilBERT, XLNet(117M), ALBERT, ELECTRA and ERNIE. In addition, probabilistic deep learning based classifier models like Multinomial Naïve Bayes (MNB) and Bidirectional LSTM-Conditional Random Field (BiLSTM-CRF) were also evaluated. For evaluation, metrics such as F1-score, recall, precision and macro-average values were used, along entity-level evaluation. Overall, transformer-based models outperformed others, achieving higher score on evaluation metrics with strong performance at the entity level. The proposed fully fine-tuned XLNet(117M) model achieved the highest training and testing accuracies of 97.1% for both. Additionally, the proposed framework outperformed other models in terms of evaluation metrics with a precision (97.1%), recall (95.9%), and F1-score (96.5%). Moreover, XLNet(117M) also obtained maximum macro average values in terms of precision (64%), recall (55%) and F1-score (58%) making it the prominent and best performing model. Since the proposed framework is trained on general purpose data, it can be used for content recommendation, information retrieval in Q&A systems. It can also serve as a base-model to use it further in domain specific tasks by transfer-learning.