Skip to content

Simple Agentic Memory for Generalist Robot Policies

Sep 2026 · arXiv.org · Vol abs/2609.36595 · 1 citation · 50 references
Computer Science

TL;DR

These results support a state-based view of robot memory: effective memory for control is not simply retained visual history, but compact task-relevant state derived from the interaction history.

Abstract

Visual-memory systems commonly retain or compress past observations. Robot control additionally requires interaction-derived state that no individual frame may explicitly represent, such as persistent identity relations, accumulated progress, or ordered procedures. We introduce Simple Agentic Robot Memory (SimpleARM), a training-free memory layer for frozen generalist robot policies. From the task instruction, SimpleARM specifies what to monitor; frozen perceptual tools maintain compact typed state online; structured access retrieves that state only when a proposed subgoal depends on history; and current-view grounding resolves recalled entities before execution. We evaluate SimpleARM on RoboMME, a benchmark of memory-dependent robot manipulation tasks that require history information no longer available in the current observation. Across all 16 tasks and three policy seeds, SimpleARM achieves 67.17% mean success, compared with 44.51% for the strongest non-oracle baseline. Matched ablations show mechanism specificity: removing relation, reference, progress, or route state produces large losses where the affected state is retrieved for control, while largely sparing other tasks. These results support a state-based view of robot memory: effective memory for control is not simply retained visual history, but compact task-relevant state derived from the interaction history.

View source

Similar papers

ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation

Comparisons within ReCAT show that the observation encoder and every-block memory conditioning are needed for this performance, and that update rules developed for efficient sequence modeling behave differently as robot memory: additive updates have the highest observed success on counting and timing, and delta-rule up...

Pankhuri Vanjani, M. Hatab, Can Mizrakli et al. · 0 citations
Preprint Sep 2026

ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control

Results show that explicit concepts provide a reusable and extensible memory interface for robot control and motivate ECoMEM, which represents task-relevant history with a reusable library of grounded concepts.

Yi-Ze Liu, Ke Wang, Mac Schwager et al. · 0 citations

MemBodied: Recurrent Associative Memory for Vision-Language-Action Models

Findings support MemBodied as a practical alternative to expanding the policy context for history-dependent manipulation, a fixed-size episodic memory with two complementary components: an associative state that records interactions across policy calls and an episode anchor that preserves a compact representation of th...

Tej Deep Pala, N. Majumder, Bryce Goh et al. · 1 citation
Preprint Oct 2026

Fast-VLA: You Only Think Once for Robot Manipulation

Vision-language-action (VLA) policies commonly place a multi-billion-parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast-VLA, which invokes a frozen 4B VLM once per...

Ke-Hong Gong, Guan-Ling Hou, Xiao-Xu Zheng et al. · 0 citations
Preprint Oct 2026

RoboDoctor: Training-Free Recursive Self-Improvement for Robot Policies

Generalist robot policies and language-model agents provide complementary capabilities, but completing a task also requires adapting their use to the consequences of physical interaction. We study execution as a locus of adaptation: feedback guides the current action and supplies evidence for revising subsequent decisi...

He-Yang Yu, Hao-Min Li, Si-Hang Li et al. · 1 citation
Preprint Oct 2026

Can a Frontier VLM Serve as a Robot Policy for Memory-Dependent Manipulation?

We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation tasks spanning temporal, spatial, object, and procedural memory. The model keeps the whole episode in context and outputs every end-effector waypoint and gripper command,...

Jack B. Jedlicki, Amish Sethi, Heng Yang · 1 citation · ⚡1

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.