Skip to content

Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies

Sep 2026 · 0 citations · 31 references
Computer Science

TL;DR

GEB is introduced, a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment and demonstrating improvements over prior memory frameworks in both multiple-choice and open-ended question answering.

Abstract

Answering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can leave physical identity unresolved: different objects may share a description, while observations of the same object remain disconnected across events. Retrieving relevant events therefore does not necessarily recover the"biography"of the particular entity a question concerns. To address this, we introduce Grounded Entity Biographies (GEB), a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment. During question answering, the biography is retrieved alongside episodic evidence, allowing the model to follow an entity through events using identity links established during memory construction. Evaluations across four benchmarks, including day-long and week-long recordings, demonstrate improvements over prior memory frameworks in both multiple-choice and open-ended question answering. On EgoLifeQA, GEB achieves 72.0% accuracy, 4.4 percentage points above the best published result. Ablations show that grounded identity association and biography reading both contribute to the gains, which additional descriptions alone do not fully recover.

View source

Similar papers

MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories

This work introduces MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader and proposes MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories.

Guang-Zhi Xiong, Xin-Yuan Zhang, Xiao Yang et al. · 0 citations
Preprint Sep 2026

ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory

ICM-Bench (Identity-Centric Memory Benchmark), which, to the best of the knowledge, is the first benchmark specifically designed to evaluate identity-centric reasoning over long video memories in multimodal agents, is introduced.

Shi-Du Ren, Yun-Ze Liu, Xingxian Liu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

EM^2Mem: Event-Centric Multimodal Memory for Large Language Models

EM^2Mem is proposed, an event-centric multimodal memory framework that binds heterogeneous evidence to event anchors during memory construction, enabling compact evidence readout over grounded multimodal events rather than modality-specific fragments.

Yijun Chen, Yangfan Zheng, Yanyang Li et al. · 1 citation
Preprint Oct 2026

EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding

Current large video-language models (LVLMs) still face challenges when dealing with long videos, mainly because frames are often processed independently, making it difficult to capture temporal dependencies across events. Although retrieval-augmented approaches have been introduced to provide additional context, most o...

Yu-Hao Qin, Jun-Bo Wang, Yu-Ke Li et al. · 0 citations
#artificial intelligence Preprint Sep 2026

TempCloze: Can Video-LLMs Identify the Missing Middle?

Temporal reasoning benchmarks for Video-LLMs are often mediated by language, leaving room for linguistic shortcuts from option wording, answer correlations, or language priors, so TempCloze is introduced, a video cloze benchmark for evaluating visual temporal reasoning in Video-LLMs.

Wen-Qi Pei, Henry Hengyuan Zhao, Yi-Lai Liu et al. · 0 citations
Preprint Sep 2026

EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding

Real-time video understanding requires incrementally maintaining a memory of streaming content, and optimizing this requires dense process signals. On-Policy Self-Distillation (OPSD), which lets one model serve as both teacher and student with the teacher receiving additional privileged information such as the question...

Yu-Wei Miao, Xue-Sheng Zhang, Wen-Hao Zou et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.