Nov 2026· Proceedings of the 34th ACM International Conference on Multimedia· pp. 4543-4552· 0 citations· 50 references
TL;DR
Memory-Enhanced Incongruity Reasoner (MEIR), a novel framework for multimodal sarcasm detection and explanation, is proposed, which is the first unified framework to jointly perform multimodal sarcasm detection and natural language explanation generation.
Abstract
Multimodal sarcasm detection has emerged as a critical research direction, and recent multimodal large language models (MLLMs) have opened new possibilities toward deeper affective reasoning in conversational video scenarios. However, existing approaches still suffer from two key challenges: data-level annotation deficiency and architecture-level context entanglement, where the former provides only utterance-level binary labels while neglecting fine-grained reasoning annotations, and the latter entangles contextual history with target delivery, preventing explicit identification of multimodal conflicts. To address these issues, we first introduce Multimodal Understanding and Sarcasm Explanation DataSet (MUSE-Set), a bilingual video dataset with four-dimensional explanatory annotations decomposing sarcasm reasoning into character descriptions, situational contexts, incongruity analysis, and intended meanings. Building on this, we propose Memory-Enhanced Incongruity Reasoner (MEIR), a novel framework for multimodal sarcasm detection and explanation. MEIR employs a cognitively inspired dual-phase architecture: an episodic memory generator first compresses context videos into structured character-tracked memories, which an incongruity reasoner then contrasts with target utterance cues to explicitly identify conflicts and generate fine-grained explanations. To the best of our knowledge, MEIR is the first unified framework to jointly perform multimodal sarcasm detection and natural language explanation generation. Extensive experiments demonstrate that MEIR achieves best performance in both tasks, consistently outperforming strong MLLMs including Gemini-3-Flash and GPT-4o.
Multimodal Sarcasm Detection (MSD) identifies sarcastic intents by jointly comprehending visual and textual modalities, providing deeper contextual understanding than conventional text-only detectors. However, it remains challenging as it relies on capturing subtle, fine-grained pragmatic incongruities across heterogen...
Rui-Peng Liu, Tian-Yuan Yu, Liang Bai et al.· Proceedings of the 34th ACM...· 0 citations
Multimodal Empathetic Response Generation (MERG) integrates textual, visual, and acoustic signals to synthesize nuanced affective responses. While Large Language Models (LLMs) demonstrate strong capabilities in emotional reasoning, they still suffer from reasoning redundancy and contextual prolixity. Applying static, h...
Xinzehua Li, Cheng-Ji Wang, Wang-Da Zhu et al.· Proceedings of the 34th ACM...· 1 citation
Tabletop role-playing games present a challenging domain for artificial intelligence, particularly for reasoning about emotional shifts and narrative significance in long-form actual-play sessions. Players improvise shared stories over hours of dialogue, performance, rules negotiation, and social reaction, while the si...
Gianlucca Lodron Zuin, Túlio Corrêa Loures, Eduardo Teixeira de Moura et al.· Proceedings of the AAAI Conf...· 0 citations
Understanding humor in videos remains an underexplored challenge for Multimodal Large Language Models (MLLMs). It demands the integration of fine-grained multimodal perception, temporal narrative reasoning, and sociocultural knowledge, which existing works have not fully explored. Consequently, the absence of high-qual...
Xue-Yan Wang, Di Yang, Kai-Wen Wei et al.· Proceedings of the 34th ACM...· 0 citations
This work proposes LongEmo, a novel memory-augmented agentic framework designed to tackle the immense challenges of long-range affective reasoning, and achieves state-of-the-art performance, demonstrating the efficacy of its event-centric memory architecture.
Shuo Zhang, Yifan Zhou, Han-Yu Wang et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.