Skip to content
Book Open access

Beyond Surface Cues: Multimodal Implicit Emotion Understanding with a New Dataset, Reasoning Framework, and Benchmark

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 39 references

Abstract

Implicit emotion understanding addresses a fundamental yet underexplored challenge: observable expressions do not always faithfully reflect underlying emotional states, requiring models to infer latent emotions from limited multimodal signals. Existing approaches largely assume direct correspondence between overt cues and true emotions, while alternatives often rely on physiological signals or extensive context, limiting practicality. To address these limitations, we present (i) a new task formulation under real-world constraints; (ii) a benchmark built on the Multimodal Implicit Emotion (MIE) dataset and a unified evaluation protocol; and (iii) EmotionReasoner, an LLM-based framework for reasoning beyond surface cues. MIE covers video, audio, and text, with annotations for both explicit and implicit emotions. A context-grounded verification mechanism reduces labeling ambiguity and improves annotation reliability. The benchmark evaluates explicit and implicit emotions through grouped metrics and fine-grained category-level analysis. EmotionReasoner performs hierarchical reasoning from global perception and local anomaly detection to structured evidence organization and implicit emotion inference. Experiments show that current multimodal large language models exhibit systematic deficiencies in implicit emotion understanding, whereas EmotionReasoner substantially improves performance and robustness. The appendix and source code are available at https://github.com/WKH186/MIE.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.