Instance Weight-Based Training: Paradigm Shift in Mini-Batch Training for Multiclass Classification in Natural Language Processing
TL;DR
A novel Instance Weight Estimation Model (IWEM) that calculates the weight of each instance in the given training data to improve model performance and identify the causes of high-weight instances in short-text emotion recognition and natural language inference, representative classification tasks in natural language processing.
Abstract
This paper presents a novel perspective on training methods that diverges from traditional fine-tuning approaches for deep learning-based natural language processing models by considering the weight of each instance. Specifically, we propose a novel Instance Weight Estimation Model (IWEM) that calculates the weight of each instance in the given training data. Using IWEM, we identify high-weight instances, which are challenging samples in multiclass classification tasks, and retrain the model with these instances. Additionally, through our analysis of high-weight instances, we identify the causes of these instances in short-text emotion recognition and natural language inference, representative classification tasks in natural language processing. Finally, to improve model performance, we augment the high-weight instances corresponding to each cause using a Large Language Model (LLM) and retrain the model. In short-text emotion recognition, the proposed model results in a 14% improvement in the $F_{1}$ -score compared to existing models, with an additional 5.3% enhancement achieved through high-weight instance augmentation. When applied to natural language inference using the KLUE NLI dataset, the proposed model demonstrates a 2% improvement in accuracy compared to existing models, with an additional 0.5% enhancement achieved by augmenting the high-weight instance.