Aug 2026· 2 citations· ⚡ 1 influential· 41 references
Computer Science
TL;DR
Evaluation of state-of-the-art open-source and closed-source MLLMs reveals that current MLLMs function as lossy summarizers rather than faithful memorizers, highlighting the need for architectures with genuine long-term spatiotemporal memory.
Abstract
Recent advances in Multimodal Large Language Models (MLLMs) have led to substantial progress in video understanding, accompanied by a growing number of long video benchmarks. However, existing benchmarks rely predominantly on web-sourced videos that lack inter-clip spatiotemporal continuity, making it difficult to assess whether models can maintain consistent memory across days or weeks of real-world experience. We introduce EgoMonth, the first month-level egocentric video understanding benchmark. EgoMonth comprises over 300 hours of first-person daily-life recordings from 20 participants spanning 20 to 120 days, paired with 1,443 human-crafted multiple-choice question-answer pairs. We design a cognitively grounded 14-task evaluation framework organized into three hierarchical cognitive levels: Schema Consolidation, Episodic Indexing, and Cascading Reasoning. Evaluation of state-of-the-art open-source and closed-source MLLMs reveals that even the best-performing model, Gemini 2.5 Pro, achieves only 71.8% macro-average accuracy, remaining 22.4 percentage points below the corrected human baseline of 94.2%. Several models perform near or below the 25% chance level on tasks such as Route Reasoning, Cross-view Spatial Reasoning, and Direction Judgement, while even the strongest closed-source model remains substantially below human performance. These results indicate that current MLLMs function as lossy summarizers rather than faithful memorizers, highlighting the need for architectures with genuine long-term spatiotemporal memory.
A cross-domain egocentric video question answering benchmark designed to evaluate whether multimodal large language models can generalize beyond common daily-life scenarios, and two official Codabench tracks.
Yu-Qian Fu, Tianwen Qian, Yanjun Li et al.· 0 citations
Light-Omni is introduced, a multimodal agent framework for reflexive and lightweight video understanding that achieves semantically aligned retrieval and reflexive responses while avoiding iterative reasoning and serves as a memory system to enhance both the performance and efficiency of existing MLLMs.
Chang Nie, Jiaju Wei, Jun-Lan Feng et al.· 0 citations
To facilitate smart wearable devices or human-like robotics with real-time first-person perspective perception ability, recent researchers proposed the Egocentric Online Action Segmentation (EOAS) task. It requires models to recognize what is happening in egocentric streaming videos and discriminate the starting and en...
Xun Jiang, Xing Xu, Chong Liu et al.· IEEE Transactions on Image P...· 1 citation
ReflectWorld-MM is proposed, an entity-oriented multimodal memory system for open-ended video streams that achieves the best accuracy on all six long-video and lifelong-memory benchmarks, outperforming strong memory agents and a frontier model.
Xiaokang Ma, Yifan Sun, Zhihong Jin et al.· 0 citations
Inspired by Multiple-Trace Theory in cognitive psychology, this work revisit long video understanding from a probe-echo perspective, in which human episodic memories are activated and integrated in parallel, and proposes ProEchoMem, a cognitive-inspired framework that simulates the probe-echo mechanism.
Derong Xu, Yanxin Chen, Wanyu Wang et al.· Annual International ACM SIG...· 0 citations
GlanceWAM is introduced, which decouples imagination from control within a single video DiT: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate purely in latent space without blo...
Lin-Han Wang, Zi-Jian An, Mingyuan Zhang et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.