TFP: Temporally Conditioned Memory-Fusion Policies for Visuomotor Learning

Yushen Liang Yue Peng Baosheng Jin Tianluo Zhang Xinyue Zhang Shuyi Zhou Zhuoran Chen Xinqiao Liu Shenji Wan

Jul 2026 · arXiv.org · Vol abs/2607.08283 · 0 citations · 35 references

Computer Science

TL;DR

Temporally Conditioned Memory-Fusion Policies (TFP), a lightweight memory-action framework for VLA backbones, is introduced and suggests that compact, event-sensitive memory dynamics can improve VLA policies under occlusion, visual perturbation, and stage-dependent task structure.

Abstract

Vision--Language--Action (VLA) policies such as $\pi_{0.5}$ and OpenVLA perform well on many manipulation tasks, but they are often reactive: the next action is predicted from the current observation, instruction, and proprioceptive state. This assumption breaks down in stage-dependent manipulation, where visually similar states may require different actions depending on latent task progress and previous interaction outcomes. We argue that such tasks require not only memory, but dynamics-aware belief updates: the policy should preserve task progress during stable or occluded phases and revise its belief near contact, release, or subgoal transitions. We introduce Temporally Conditioned Memory-Fusion Policies (TFP), a lightweight memory-action framework for VLA backbones. TFP maintains an episode-local task-progress belief with Liquid Time-Constant dynamics and injects the updated belief directly into the flow-matching action decoder through adaptive modulation. This lets temporally accumulated context shape the generated action chunk, rather than serving only as passive history context. With a 3.3B-parameter model, TFP improves the average success rate from 96.9% to 98.75% on LIBERO and from 91.4% to 93.77% on LIBERO-plus. On the memory-focused MIKASA ShellGameTouch diagnostic, TFP achieves success up to 75.0%. Mechanistic analyses show that write-gain changes near manipulation events are about 6 times larger than far non-event phases, and hidden-state interventions show that the belief causally modulates generated action chunks. These results suggest that compact, event-sensitive memory dynamics can improve VLA policies under occlusion, visual perturbation, and stage-dependent task structure.

DOI arXiv Semantic Scholar

View source

Similar papers

Jul 2026

WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory

WorldScape Policy 2.0 is introduced, a controllable WAM with reasoning-augmented long short-term memory and fine-grained instruction following and in-context adaptation that demonstrates superior capabilities in long-horizon autonomous planning, fine-grained instruction following and in-context adaptation.

Hai-Sheng Su, Zong-Dai Liu, Xin Jin et al. · arXiv.org · 2 citations

Preprint Aug 2026

Retrieve in Time, Correct in Frequency

Frozen vision-language-action (VLA) policies generate temporally extended action chunks, but long-horizon manipulation remains vulnerable to accumulated execution error and visual aliasing across task stages. Successful rollouts provide useful corrective evidence, yet current frame retrieval can return progress-misaligned actions,while direct replay or time-domain fusion can overwrite the reactive structure of the policy proposal. We introduce Retrieve in Time, Correct in Frequency (RTCF), a training-free test-time correction framework that improves frozen VLA performance with low model-side overhead.RTCF separates which experience to retrieve from which part of its action to transfer. Progressive Memory Alignment (PMA) causally aligns the growing visual execution history with complete successful trajectories through incrementally updated monotonic frontiers, jointly identifying a relevant memory and the current aligned memory position without stage labels. From the aligned action chunk,RTCF transfers a coefficient-wise-clipped low-frequency residual on motion channels. Higher-frequency components and gripper decisions remain inherited from the frozen policy. Across four LIBERO suites and 2,000 episodes per condition, RTCF raises aggregate success from 86.4% to 88.4% and improves LIBERO-Long from 61.6% to 68.6%.These gains require no parameter updates, repeated VLA inference, or additional GPU resources: correction can be performed on the client CPU after a single policy invocation, and the median latencies sum to only 10.99 ms per action chunk

Yuze Fan, Yue Cao, Peng-Jie Gao et al. · 0 citations

#machine learning Preprint Aug 2026

LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Prediction

LM-X is introduced, which organizes prediction across task, event, and motor scales without claiming anatomical correspondence and shows that explicit multi-timescale predictive state can strengthen control while exposing interpretable internal estimates.

Jin Lou, Zhi Jing, A. Chen et al. · 0 citations

Preprint Aug 2026

Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning

Temporal GRPO addresses the problem of trajectory-level credit aliasing in post-train VLA policies by constructing detectable task stages, aligning each rollout with stage-specific action intervals, and comparing only rollouts that have entered the same stage.

Yao Zhou, Hang Gao, Fengge Wu et al. · 0 citations

Preprint Aug 2026

Skills in Weights, Memory in Code: Hybrid Learning for Memory-Dependent Robot Manipulation

HyMeS, a hybrid learning framework that leverages the reasoning and memory-management capabilities of coding agents to steer a Markovian VLA for memory-dependent manipulation, is proposed, enabling data-efficient compositional generalization.

Yunhao Zhao, Zhenyang Ni, Haoyang Chen et al. · 0 citations

Jul 2026

MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation

MemVLN is proposed, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS) and introduces Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency.

Yuqi Liu, Sheng-Ju Qian, Tianyuan Qu et al. · arXiv.org · 0 citations