Skip to content

Multimodal sentiment analysis based on image captions and aspect-guided soft prompts

Jul 2026 · Multimedia Systems · Vol 32 · 0 citations · 38 references

TL;DR

This work proposes a novel framework that employs the pre-trained vision-language model BLIP (Bootstrapping Language-Image Pre-training) to generate descriptive image captions and introduces an aspect-guided soft prompt mechanism that enables dynamic interaction between aspect terms and multimodal features, thereby mitigating the effects of structural irregularities.

View source

Similar papers

Aug 2026

Aspect-guided dual-branch fusion network for multimodal aspect-based sentiment analysis

An Aspect-guided dual-branch fusion network (ADFN) to enhance sentiment prediction by incorporating external knowledge and integrating coarse and fine information is proposed, which incorporates syntactic dependency information to complement and enrich the textual semantic representations.

Bin Song, Wenjing Liu, Zhi Liang et al. · 0 citations
Open access Aug 2026

Tweet-Image-Caption conditioned diffusion model for multimodal Aspect-Based sentiment analysis

TicCondDiffusion is the first diffusion-based framework specifically designed for MABSA, offering a new direction for this task, and achieves competitive performance compared with recent baseline approaches under the end-to-end setting.

Hao-Mei Jia, Wen-Min Wang, Wen-Feng Huang et al. · 0 citations
#small language model Preprint Aug 2026

Multi-Granularity Sentiment Integration for LLM-Based Multimodal Sentiment Analysis

MGSI first encodes audio and visual streams at short-, medium-, and long-range temporal scales, preserving both local variations and global affective trends, and applies polarity- and intensity-aware enhancement to better handle ambiguous and near-neutral samples.

Shanshan Lin, Yuesheng Wu, Chao Chen et al. · 0 citations
Open access Aug 2026

Image–Text Multimodal Sentiment Analysis with Large Model-Generated Descriptive Semantics and Difference-Aware Gated Fusion

Experimental results on the MVSA-Single and MVSA-Multiple datasets show that the proposed method improves performance in image–text multimodal sentiment classification, thereby validating the effectiveness of combining semantic enhancement with difference-aware modeling.

Hengyuan Zhang, Aizihaierjiang Yusufu, Jiang Liu et al. · 0 citations
Open access Jul 2026

IPMMSA: MODELLING AN IMPROVED PROMPT-BASED MULTI-MODAL SENTIMENT ANALYSIS OVER FASHION DATASETS

This work suggests an improved prompt-based multi-modal sentiment analysis (IPMMSA) strategy that incorporates multi-view and diversified knowledge augmentation that yields robust and expressive multimodal embedding’s to boost aspect-based sentiment analysis performance during multimodal integration with multi-modal fa...

M. Yuvaraja, C. Kumuthini · 0 citations
Aug 2026

MagXCL: enhanced multimodal adaptation gate and cross-modal contrastive learning for multimodal sentiment analysis

MagXCL, a unified framework designed to improve multimodal integration through more effective interaction between verbal and non-verbal modalities, is proposed, demonstrating the effectiveness of combining AMag with CrossCL to produce more accurate and robust multimodal sentiment predictions.

Duc-Duy Duong, Cam-Van Thi Nguyen, Duc-Trong Le · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.