A Cross-Attention-Based Framework for Multimodal Sentiment Analysis

Learning Adaptive Cross-Modal Interactions for Multimodal Sentiment Analysis

Multimodal sentiment analysis aims to integrate heterogeneous textual, visual, and acoustic information for effective emotion understanding. However, existing methods often suffer from insufficient cross-modal interaction modeling, limited adaptability in multimodal fusion, and inadequate suppression of modality-specific noise under complex conversational scenarios. To address these challenges, this paper proposes a framework for learning adaptive cross-modal interactions for multimodal sentiment analysis. The proposed framework consists of three stages: modality-aware preprocessing, heterogeneous representation learning, and adaptive multimodal fusion. First, a unified preprocessing strategy is designed to improve cross-modal consistency through textual normalization, speaker-aware visual alignment, and utterance-level acoustic representation enhancement. Second, modality-specific encoders are constructed to capture complementary semantic, spatial, and utterance-level acoustic characteristics from textual, visual, and acoustic modalities, respectively. Third, an adaptive fusion framework is introduced to explicitly model cross-modal interactions, dynamically estimate the importance of different modality combinations, and further calibrate discriminative feature channels through channel attention. By jointly performing modality-level interaction learning and channel-wise feature refinement, the proposed framework effectively enhances multimodal representation capability for sentiment classification. Extensive experiments conducted on the CMU-MOSI and MELD benchmark datasets demonstrate that our framework consistently outperforms previous methods. In particular, the proposed model achieves 90.27% accuracy and 90.26% F1-score on CMU-MOSI, together with 66.57% accuracy and 66.21% F1-score on MELD. Additional ablation studies and qualitative analyses further validate the effectiveness of the proposed preprocessing strategy, modality-specific representation learning, and adaptive fusion mechanism.

MagXCL: enhanced multimodal adaptation gate and cross-modal contrastive learning for multimodal sentiment analysis

Hierarchical Global-Local Interaction and Refinement for Multimodal Sentiment Analysis

Multimodal Sentiment Analysis (MSA) aims to integrate text, audio, and visual modalities to achieve accurate sentiment modeling. Existing methods often rely on shallow interaction structures, making it difficult to jointly capture fine-grained local dynamics and high-level semantic dependencies. In addition, dominant modalities may suppress the learning of weaker modalities, leading to insufficient cross-modal semantic alignment. To address these issues, we pro-pose a Hierarchical Global-Local Interaction and Refinement framework for Multimodal Sentiment Analysis (HGLIR). Specifically, a Modality Dropout strategy is first introduced at the input stage to alleviate over-reliance on a sin-gle modality and improve robustness. Based on this, a Hierarchical Local Inter-action (HLI) module models multimodal sequences through a multi-layer pro-gressive structure to capture local dynamic features at different semantic levels. Within the HLI module, a Cross-Modal Synergistic Learning (CMSL) mecha-nism explicitly models cross-modal semantic consistency and gradually aligns information during interaction. Furthermore, a Global Representation Refine-ment (GRR) module introduces learnable global representations and iteratively updates them in a multi-layer structure to aggregate long-range semantic de-pendencies and form stable high-level semantic representations. Experimental results on CMU-MOSI and CMU-MOSEI demonstrate the effectiveness of the proposed framework across multiple evaluation metrics.

Contrastive Bidirectional Cross-Modal Attention Framework for Enhanced Multimodal Sentiment Analysis

The high rate of social media content development causes an increase in multimodal data, such that modeling relationships between visual and textual data is challenging. Nevertheless, most of the available methods cannot capture fine-grained text-to-visual or visual-to-text interaction, resulting in lower sentiment performance. A Contrastive Bidirectional Cross-Modal Attention (C-BCMA) model is presented to enhance the correspondence of textual and visual representations by acquiring a common latent space. An attention method inspired by CLIP is utilized to produce robust cross-modal latent features to enhance their joint representation. Textual features are derived using ALBERT, whereas EfficientNet-B2 is applied to obtain visual representations. Interactions between modalities are learned using a multi-head attention mechanism. Textual and visual information is handled jointly during learning. This helps reduce gaps between the two modalities. This enables the model to process various semantic cues at once. Contrastive learning is used in the model to align similar text-image pairs and to separate unrelated text-image pairs so that better multimodal representations are achieved. The model has a better performance than baseline approaches on both single and multiple annotation versions of MVSA datasets. It achieves better performance across various evaluation metrics. Less obvious expressions like sarcasm and implicit sentiment are handled more effectively in this work, improving interpretation in multimodal sentiment analysis of social media data.

TGHIN: text-guided hyper-modality interaction network for multimodal sentiment analysis

MULTI-MODAL TRANSFORMER ARCHITECTURE WITH CROSS-ATTENTION FUSION FOR ROBUST AUDIO-VISUAL SENTIMENT ANALYSIS

Multimodal sentiment analysis (MSA) has gained significant attention due to its ability to integrate heterogeneous information from audio, visual, and textual modalities. However, existing transformer-based fusion methods often suffer from reduced robustness when one or more modalities are corrupted or partially unavailable. This paper presents a Multi-Modal Transformer Architecture with Cross-Attention Fusion (MMT-CAF) for robust audio-visual sentiment analysis. The proposed framework combines modality-specific transformer encoders, bidirectional cross-attention, and a reliability-aware fusion mechanism that dynamically adjusts the contribution of each modality according to its estimated reliability. The framework was evaluated on the CMU-MOSI and CMU-MOSEI benchmark datasets and compared with representative transformer-based methods, including Adaptive Modality Weighting, RAFT, and CITN-DAF. Experimental results demonstrate that MMT-CAF achieves superior sentiment classification performance while maintaining higher robustness under noisy audio, visual occlusion, and missing-modality scenarios. Ablation studies further confirm the effectiveness of the proposed cross-attention and reliability-aware fusion modules in improving multimodal representation learning. The proposed architecture provides an effective and interpretable framework for robust multimodal sentiment analysis and offers a promising foundation for real-world affective computing applications.