2026· Poster Volume 0008 The 2026 Twenty-Second International Conference on Intelligent Computing July 23-26, 2026 Toronto, Canada· pp. 531-559· 0 citations
TL;DR
Consistency-Aware Gated Fusion (CAGF), a lightweight and fusion module tailored to Mamba-based architectures that achieves state-of-the-art performance, outperforming strong multimodal baselines such as CLIP, MISA, DLF, AoM, and SFTTR, while remaining more efficient and interpretable.
Abstract
Multimodal sentiment analysis has attracted increasing attention due to the prevalence of text-image content on social media. A central challenge is to design fusion mechanisms that are both expressive and parameter-efficient, especially for small-scale datasets where heavy cross-modal attention can easily overfit. In this paper, we present Consistency-Aware Gated Fusion (CAGF), a lightweight and fusion module tailored to Mamba-based architectures. Our key idea is to exploit Mamba's bidirectional scanning mechanism: forward and backward hidden states from text and image encoders are concatenated to form enhanced representations, and a cosine-based semantic consistency score is computed between modalities. This score is then passed through a fixed sigmoid gate to adaptively weight text and image features, without introducing any additional learnable parameters. CAGF is plug-and-play compatible with dual-stream Mamba encoders and incurs negligible computational overhead compared with attention-based fusion. Experiments on the MVSA-Single dataset show that CAGF achieves state-of-the-art performance (Acc=82.54%, F1=84.82%), outperforming strong multimodal baselines such as CLIP, MISA, DLF, AoM, and SFTTR, while remaining more efficient and interpretable. Extensive ablations and sensitivity analyses further validate that bidirectional scanning, enhanced representations, and consistency-aware gating are all critical to the observed gains.
DualScope is proposed, a novel model that combines a global-local fusion strategy with bidirectional image-text generation for semantically consistent data augmentation and introduces both label contrastive learning and data contrastive learning to align heterogeneous modalities and enhance model robustness.
Bing Zhang, Junteng Wang, Bin Sun et al.· Memetic Computing· 0 citations
In multimodal sentiment analysis, textual, acoustic, and visual modalities often contain redundant and noisy information. Such information increases model complexity and weakens core sentiment representations, degrading accuracy and robustness. To address this issue, we propose CLIBN, a multimodal sentiment recognition network based on contrastive learning and information bottleneck. First, we design a sentimentintensity-aware contrastive learning strategy. It constructs positive and negative pairs according to sentiment intensity distances and assigns adaptive weights to different pairs, enabling the model to capture fine-grained sentiment differences. Second, we introduce a hierarchical information bottleneck module. It treats text as the primary modality and progressively integrates complementary cues from acoustic and visual modalities, while preserving task-relevant semantics and suppressing redundant information. Experimental results on CMU-MOSI and CMU-MOSEI show that CLIBN achieves superior performance. Specifically, Acc-2 reaches 87.8% and 86.7%, and F1-Score reaches 87.8% and 86.6% on the two datasets, respectively. These results demonstrate the effectiveness of CLIBN for multimodal sentiment representation learning.
Xu Meng, Yi Zhang, Yang Li· 2026 8th International Confe...· 0 citations
The present study proposes such a multimodal sentiment analysis framework with attention-enhanced properties, a combination of ResNet50 and Convolutional Block Attention Module (CBAM), a textual encoder with BERT, and refinement of relational features via Graph Neural Networks (GNN). The model is designed to address the vulnerability of uni-modal sentiment analysis and integrate related visual and textual evidence. CBAM enhances the visual feature representation with the assistance of channel and spatial attention, but BERT proposes text embeddings in their context. Another model similar to multimodal representations and similarity-based sampling relationships is a Graph Neural Network. It is experimentally demonstrated that the proposed framework is characterized by a total classification accuracy of 0.7676 compared to baseline and conventional attention-based models. The additional outcomes of Precision show that enhanced retrieval performance was obtained, which highlights the fact that multimodal fusion that is strengthened by mental attention can be effective in the sentiment classification task.
K. Mounika, B. V. RamNaresh Yadav· International journal of com...· 0 citations
MGSI first encodes audio and visual streams at short-, medium-, and long-range temporal scales, preserving both local variations and global affective trends, and applies polarity- and intensity-aware enhancement to better handle ambiguous and near-neutral samples.
Shanshan Lin, Yuesheng Wu, Chao Chen et al.· 0 citations
Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating heterogeneous modalities such as language and acoustic signals. Despite recent progress, two key challenges remain: (1) inter-modal inconsistency, where different modalities may convey conflicting sentiment cues, and (2) intra-modal feature ambiguity caused by noise and subtle emotional variations. To address these issues, we propose RegCal-Net, a register-augmented and self-calibrated framework for bimodal sentiment analysis. First, we introduce a Register-Augmented Self-Attention (RASA) mechanism that appends learnable register tokens along the sequence dimension to provide auxiliary global anchors for each modality. Second, we design a Self-Calibrated Fusion (SCF) module that dynamically evaluates fused feature reliability through an auxiliary score-guided gating strategy, enabling adaptive suppression of unreliable signals during multimodal integration. Extensive experiments on two widely used benchmarks, CMU-MOSI and CMU-MOSEI, together with CMU-MOSI encoder-controlled baselines and a supplementary video-subset validation, demonstrate that RegCal-Net achieves competitive performance across multiple evaluation metrics, particularly improving fine-grained sentiment classification and reducing prediction error. These results indicate that combining register-based representation stabilization with quality-aware fusion provides an effective solution for robust multimodal sentiment analysis.
Bin Xu, Wei-Yang Wang, Ao Ding· Multimedia Systems· 0 citations
A Global Tri-Modality Transformer (GTMT) that first performs parallel fusion of the three modalities and then conducts deep integration guided by the textual modality to achieve the cross-modal semantic alignment and correlation, significantly improving the global unified fusion effectiveness of the tri-modality information.
Zelong Li, Shu-Hua Lu, Fang Cui et al.· Neural Networks· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
