Linguistically Informed Machine Learning for Gujarati–English Code-Mixed Sentiment Classification: A Comparative Study of Feature Fusion Strategies

Abstract

Code-mixed text, in which words from multiple languages are used within the same sentence, is common on social media platforms and poses significant challenges for conventional natural language processing techniques. This study focuses on five-class sentiment classification of of Gujarati–English code-mixed text. The dataset consists of 5 sentiment classes ranging from extremely negative to extremely positive distributed over 44,672 sentences. A linguistically informed framework is proposed that incorporates word-level annotations, including language identity, sentiment polarity, and intensifier presence, generated using a multi-task fine-tuned DistilBERT tagger. These annotations are aggregated into sentence-level handcrafted features and combined with conventional text representations, namely Bag of Words (BoW), Term Frequency–Inverse Document Frequency (TF-IDF), Word2Vec, and FastText. The proposed framework is evaluated using a late fusion approach based on out-of-fold (OOF) stacking and is compared with early fusion, where features are directly concatenated, and with embedding-only baselines. The statistical significance of performance differences is assessed using McNemar's test. Seven machine learning classifiers—Logistic Regression (LR), Multinomial Naïve Bayes (MNB), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Decision Tree (DT), Random Forest (RF), and Extreme Gradient Boost(XGB)—are evaluated. The hyperparameters of all classifiers are optimized using GridSearchCV and then kept fixed throughout the experiments to ensure a fair comparison. Class imbalance is addressed using cost-sensitive learning through class-weight adjustment. The experimental results show that early fusion of linguistic and text features consistently outperforms the embedding-only and late fusion approaches for most classifier–representation combinations. The best-performing model is XGB with FastText under the early fusion framework, achieving an accuracy of 0.78 and a macro F1-score of 0.74 on the unseen test set. Overall, this work demonstrates that incorporating explicit linguistic information, including language identity, sentiment polarity, and intensifier information, improves sentiment classification of Gujarati–English code-mixed text. In addition, it provides a comprehensive and statistically validated comparison of feature integration strategies for sentiment analysis in low-resource, code-mixed language settings.