Skip to content
Book Open access

MEIR: Memory-Enhanced Incongruity-Aware Reasoning for Multimodal Sarcasm Detection and Explanation

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · pp. 4543-4552 · 0 citations · 50 references

TL;DR

Memory-Enhanced Incongruity Reasoner (MEIR), a novel framework for multimodal sarcasm detection and explanation, is proposed, which is the first unified framework to jointly perform multimodal sarcasm detection and natural language explanation generation.

Abstract

Multimodal sarcasm detection has emerged as a critical research direction, and recent multimodal large language models (MLLMs) have opened new possibilities toward deeper affective reasoning in conversational video scenarios. However, existing approaches still suffer from two key challenges: data-level annotation deficiency and architecture-level context entanglement, where the former provides only utterance-level binary labels while neglecting fine-grained reasoning annotations, and the latter entangles contextual history with target delivery, preventing explicit identification of multimodal conflicts. To address these issues, we first introduce Multimodal Understanding and Sarcasm Explanation DataSet (MUSE-Set), a bilingual video dataset with four-dimensional explanatory annotations decomposing sarcasm reasoning into character descriptions, situational contexts, incongruity analysis, and intended meanings. Building on this, we propose Memory-Enhanced Incongruity Reasoner (MEIR), a novel framework for multimodal sarcasm detection and explanation. MEIR employs a cognitively inspired dual-phase architecture: an episodic memory generator first compresses context videos into structured character-tracked memories, which an incongruity reasoner then contrasts with target utterance cues to explicitly identify conflicts and generate fine-grained explanations. To the best of our knowledge, MEIR is the first unified framework to jointly perform multimodal sarcasm detection and natural language explanation generation. Extensive experiments demonstrate that MEIR achieves best performance in both tasks, consistently outperforming strong MLLMs including Gemini-3-Flash and GPT-4o.

Read PDF

Similar papers

Book Open access Nov 2026

MIRD: Multimodal Implicit Reasoning Distillation for Efficient Sarcasm Detection

Multimodal Sarcasm Detection (MSD) identifies sarcastic intents by jointly comprehending visual and textual modalities, providing deeper contextual understanding than conventional text-only detectors. However, it remains challenging as it relies on capturing subtle, fine-grained pragmatic incongruities across heterogen...

Rui-Peng Liu, Tian-Yuan Yu, Liang Bai et al. · 0 citations
Book Open access Nov 2026

Towards Human-like Empathy: Confidence-Driven Fast-Slow Reasoning for Multimodal Empathetic Response Generation

Multimodal Empathetic Response Generation (MERG) integrates textual, visual, and acoustic signals to synthesize nuanced affective responses. While Large Language Models (LLMs) demonstrate strong capabilities in emotional reasoning, they still suffer from reasoning redundancy and contextual prolixity. Applying static, h...

Xinzehua Li, Cheng-Ji Wang, Wang-Da Zhu et al. · 1 citation
Conference Nov 2026

Finding Moments Worth Remembering: Multimodal Evidence and Multi-Stage Reasoning for Illustrating Actual-Play TTRPG Sessions

Tabletop role-playing games present a challenging domain for artificial intelligence, particularly for reasoning about emotional shifts and narrative significance in long-form actual-play sessions. Players improvise shared stories over hours of dialogue, performance, rules negotiation, and social reaction, while the si...

Gianlucca Lodron Zuin, Túlio Corrêa Loures, Eduardo Teixeira de Moura et al. · 0 citations
Book Open access Nov 2026

ComedyQA: Revealing Visual, Cultural, and Reasoning Challenges in Video Humor Understanding

Understanding humor in videos remains an underexplored challenge for Multimodal Large Language Models (MLLMs). It demands the integration of fine-grained multimodal perception, temporal narrative reasoning, and sociocultural knowledge, which existing works have not fully explored. Consequently, the absence of high-qual...

Xue-Yan Wang, Di Yang, Kai-Wen Wei et al. · 0 citations

LongEmo: Towards Emotion Understanding and Reasoning in Long Videos

This work proposes LongEmo, a novel memory-augmented agentic framework designed to tackle the immense challenges of long-range affective reasoning, and achieves state-of-the-art performance, demonstrating the efficacy of its event-centric memory architecture.

Shuo Zhang, Yifan Zhou, Han-Yu Wang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.