BERT-based short text sentiment classification model integrating dynamic pruning and multiscale attention

Abstract

To address the problems of large parameter scale, low inference efficiency, and insufficient capture of local sentiment cues in the original BERT model for short-text sentiment classification, this paper proposes an improved BERT-based sentiment classification model integrating dynamic pruning and multi-scale attention. Unlike existing lightweight BERT methods that mainly emphasize parameter compression and inference acceleration, the proposed method is tailored to short-text sentiment analysis, where model compression may weaken the modelling of local sentiment words, negation patterns, and short-range semantic dependencies. To this end, a collaborative framework combining non-uniform dynamic pruning and semantic compensation is constructed. Specifically, an importance-based dynamic pruning strategy is applied to the BERT encoder layers to reduce structural redundancy and computational cost, while a multi-scale attention mechanism is introduced to jointly capture global semantic dependencies and local key sentiment information, thereby alleviating the representation loss caused by pruning. Experimental results on the IMDB and SST-2 datasets show that the proposed model achieves favourable classification performance with clear lightweight advantages. Compared with the baseline BERT, the 50% pruning scheme reduces the number of parameters by 25.6%, lowers peak memory usage by 31.2%, and increases inference speed by 68.4%, with only a slight drop in accuracy. The results demonstrate that the proposed method can effectively improve deployment efficiency while retaining competitive sentiment classification capability, providing a practical reference for short-text sentiment analysis scenarios such as e-commerce review analysis, social-media opinion monitoring, short-video comment mining, and intelligent customer service.