This work introduces MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader and proposes MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories.
Abstract
Long-term egocentric video enables personalized AI assistants to reason about daily life. However, as video histories grow to hundreds of hours spanning months or years, reprocessing raw clips for every query becomes computationally prohibitive. Memory systems offer a scalable alternative by compacting videos into text representations, but often fail on practical benchmarks: either the memory does not preserve key evidence, or the retriever fails to locate relevant entries due to retrieval competition in growing search spaces. To address these challenges, we introduce MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader. Without training or query-time video access, MemLife improves over the strongest training-free baseline by 4.6--12.0% across four long-horizon benchmarks. To further improve memory quality, we propose MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories. MemOpt consistently improves MemLife by 2.7--5.0% across different video and question distributions, with gains that generalize across writer and reader backbones and memory systems.
Long-horizon robotic manipulation fundamentally requires memory: agents must retain task-relevant information from past observations and interactions and use it to guide future actions. Existing approaches typically preserve raw historical observations, compressed visual tokens, or language summaries, but these represe...
Yun Zhang, Zhi-Yu Huang, Tian-Hui Cai et al.· 1 citation
Results support the effectiveness of caption memory for episodic reasoning over long egocentric video in wearable assistants with bounded frame budgets, growing visual-token costs, and long-context retrieval failures.
Dingli Liang, Yi Xie, Yu-Kai Huang et al.· Proceedings of the 2026 Conf...· 1 citation
Long videos often exceed the context capacity of multimodal large language models (MLLMs), while agentic search repeats costly video processing and grounding for each query. Reusable memory can amortize this cost, but must preserve information for questions unknown during construction. We propose Video-EMem, an evolvab...
Ke-Rui Chen, Jing-Lu Wang, Yun Wang et al.· 0 citations
Long-horizon robotic operation requires memory that scales with essential information rather than execution time. We present a parameter-space memory framework where each episode is stored as a separable object. A neural implicit representation is fitted to episode video from a shared, meta-learned initialization; the...
Leann Alhashishi, Christos Margadji, S. Pattinson· 0 citations
VideoTapestry is introduced, a training-free multi-agent framework that adapts a preconstructed hierarchical video memory through coarse-to-fine, query-driven refinement, preserving global context in a compact form while retaining fine-grained evidence along query-relevant branches for final reasoning.
Yu-Cheng Liu, Yu-Fei Yin, Ming-Xiao Feng et al.· 0 citations