A Lightweight DistilBERT-Attention Model for Aspect-Based Sentiment Analysis

Abstract

This paper introduces an efficient DistilBERT-Attention model for aspect-based sentiment analysis (ABSA), designed to balance classification accuracy against computational cost. Unlike general sentiment analysis, which assigns a single polarity to a complete review, ABSA identifies the product aspects discussed within individual sentences or clauses, such as design, quality, and price, and determines the sentiment polarity expressed toward each of them. The proposed model combines DistilBERT, a compact transformer encoder, with a modified aspect-focused attention layer that captures fine-grained sentiment signals efficiently. Experiments were conducted on 3,152 valid textual reviews, drawn from an initial collection of 3,259 Amazon India reviews of Titan watches published in 2024, across five aspect categories: design, quality, price, functionality, and comfort. The proposed model achieved an accuracy of 84.7% and an F1-score of 0.81. Compared with traditional baselines, it improved accuracy by 13.5 percentage points over a support vector machine and by 13.9 percentage points over logistic regression. Although BERT-base achieved a slightly higher accuracy of 86.1%, the proposed model retained approximately 98.4% of BERT-base accuracy while reducing memory consumption by about 40% and lowering relative processing time from 2.5x to 1.5x. The main contribution of the proposed model is therefore not absolute superiority over large transformer models, but an improved balance between accuracy, interpretability, and computational efficiency for resource-constrained ABSA applications.