Hybrid Multilingual Transformer and Linguistic Feature Fusion for Apology Sincerity Detection

Abstract

Identifying the sincerity of apologies in multilingual and code-switched environments is a difficult task due to the challenges posed by varying languages and cultures and the implicit nature of pragmatic markers of sincerity. In this paper, we propose a hybrid model based on XLM-RoBERTa contextual embeddings augmented with manually designed linguistic features encoding lexico-syntactic, pragmatic, and contextual clues. Our solution is validated on a collection of 5,190 apology expressions in English, Hindi and Hinglish collected using a 3-phase annotation framework based on crowdsourced human annotators and further validated using Claude – 4.6 (Cohen’s κ = 0.75). Using a two-pipeline framework, transformer-based semantic embeddings are combined with 21 manually engineered features to achieve better performance and explainability. Our experiments show that our proposed method can detect apologies’ sincerity with a weighted F1-score of 0.916, significantly outperforming the context-based XLM-RoBERTa model (F1 = 0.823) by 9.3% and the mBERT baseline model (F1 = 0.720) by 27.19%. Moreover, our model achieves high accuracy across different languages in a zero-shot setting with an F1-score ranging from 82.45% (English → Hindi) to 81.73% (English → Spanish). Some of the applications of this method are automated dispute resolution for e-commerce websites (for example, identifying insincere apologies from the vendors), calibrating mental health bots, and moderating social media websites that use multiple languages.