Predictive Sensorimotor Representation learning is introduced, a framework that learns a hierarchy of predictive representations from multimodal sensorimotor signals and integrates them into the action stream of a visuomotor policy, and is instantiate within a Vision-Language-Action model, resulting in PSR-VLA.
Abstract
Contact-rich manipulation requires policies to generate precise actions by reasoning over contact forces, robot configurations, and interaction histories beyond visual observations. Existing methods passively condition on force feedback rather than actively predicting future contact dynamics, limiting their ability to generate high-precision actions. To address this problem, we introduce Predictive Sensorimotor Representation (PSR) learning, a framework that learns a hierarchy of predictive representations from multimodal sensorimotor signals and integrates them into the action stream of a visuomotor policy. Specifically, during a pretraining stage, a multimodal Transformer is trained to learn a hierarchy of predictive representations by jointly forecasting future interaction dynamics. The learned hierarchy subsequently augments the action stream, enabling the resulting policy to exploit contact-relevant cues at multiple depths. We further instantiate PSR within a Vision-Language-Action (VLA) model, resulting in PSR-VLA, and evaluate it on six real-world contact-rich manipulation tasks. Experimental results show that PSR-VLA achieves 91.7% overall success, improving over $\pi_{0.5}$, ForceVLA-$\pi_{0.5}$, and ForceVLA2-$\pi_{0.5}$ by 30.0, 22.5, and 19.2 percentage points, respectively. These results demonstrate the effectiveness of the proposed PSR for force-aware, contact-rich manipulation. Videos of the tasks and stability tests are available at https://psr-vla.pages.dev/.
Vision-language-action (VLA) policies infer grasp actions primarily from visual observations and robot state, but do not explicitly represent the physical response observed after contact. We present MIM-VLA, a motor-feedback-based architecture that encodes recent gripper current, position, velocity, and signal validity...
Jae-young Lee, Jiyeon Koo, Taehwa Kim et al.· 0 citations
Opt2VLA is presented, a force-aware VLA framework that introduces explicit force commands at the VLA-to-control interface for humanoid whole-body manipulation and shows that explicit force conditioning enables more accurate and consistent force regulation than motion-only control, while physically grounded torque super...
Fu-Kang Liu, Yipu Chen, Jaehwi Jang et al.· arXiv.org· 1 citation
Results indicate that augmenting a fixed pretrained VLA backbone with explicit temporal and structured perceptual intermediate information provides a practical and effective way to improve robotic manipulation.
Pei-Sen Huang, Jian-Hua Hu, Jie-Ren Deng et al.· 2026 IEEE 22nd International...· 0 citations
ForeTac-VLA is proposed, a forecasting-based tactile-vision-language fusion model that predicts future tactile states to guide action generation and achieves an average success rate of 95%, outperforming the fine-tuned VLA model by 36.25 percentage points and state-of-the-art tactile-enhanced VLA baselines by over 22 p...
WorldLine is introduced, an action-driven visual simulator that decouples transferable dynamics learning from heterogeneous action grounding and makes WorldLine a scalable and efficient visual simulator for policy evaluation and embodied planning.
A mechanism-by-mechanism transition to the recurrent behavior-cloning baseline shows that replacing the predictive pathway with direct observation input produces the largest single performance drop, accounting for approximately $70\% of the endpoint gap.
Hiroki Sawada, Shunichi Kasahara· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.