Skip to content

VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining

Mar 2026 · 23 citations · 49 references
Computer Science

TL;DR

VTAM is presented, an efficient visuo-tactile world-action model for contact-rich manipulation that requires no tactile or action pretraining, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data.

Abstract

Tactile vision-language-action models (VLAs) and tactile world-action models (WAMs) are a promising new paradigm for contact-rich manipulation, but existing models either underutilize touch, rely on large-scale tactile pretraining, or incur high inference latency due to iterative video generation. We present VTAM, an efficient visuo-tactile world-action model for contact-rich manipulation that requires no tactile or action pretraining. RGB and tactile observations share a frozen video VAE and the pretrained video transformer weights. VTAM first adapts the video backbone to jointly predict future visual and tactile latents, then fine-tunes it end-to-end with a randomly initialized action expert that predicts actions, proprioceptive states, and a light flow-force proxy computed from tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). Without full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and that flow-force proxy keeps the policy from fully ignoring touch. The tactile encoder transfers zero-shot across GelSight Mini and Xense Photon sensors.

View source

Similar papers

Preprint Oct 2026

SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?

Tactile sensing provides essential contact information for robotic manipulation, yet incorporating it into pretrained vision-language-action (VLA) models remains challenging. A common concern is that simply introducing touch during task-specific fine-tuning may fail to bridge the cross-modal gap, yielding limited gains...

Chen Yang, Lin-Zhe Shi, Chang Jie Wu et al. · 1 citation · ⚡1

DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

A visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a video diffusion world model for joint visuo-tactile world modeling is presented.

Hao-Ran Yuan, Ze-Kai Wang, Bo-Ning Shao et al. · 1 citation
Sep 2026

ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation

ForeTac-VLA is proposed, a forecasting-based tactile-vision-language fusion model that predicts future tactile states to guide action generation and achieves an average success rate of 95%, outperforming the fine-tuned VLA model by 36.25 percentage points and state-of-the-art tactile-enhanced VLA baselines by over 22 p...

Zheng Tao, Xin Li, Xin Wang · 1 citation
Preprint Sep 2026

TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model

World action models improve robotic manipulation by conditioning actions on predicted futures, yet existing tactile variants largely inherit video-generation pipelines that reconstruct future tactile observations through iterative denoising. Such prediction can become unreliable under deployment drift: small changes in...

En-Yi Wang, Ming-Xin Wang, Quan Shi et al. · 2 citations
Preprint Oct 2026

OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified Sim-and-Real Framework

This work introduces OpenViTac, a visuo-tactile manipulation benchmark for evaluating robot policies across simulation and the real world, and introduces OpenVTLA, a tactile augmentation framework that combines the best-performing representation and integration strategy.

Yi-Fan Wu, Qin Li, Nan Min et al. · 2 citations · ⚡1
Preprint Oct 2026

ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction

Abstract Action-conditioned world models for contact-rich manipulation require data that captures how contact forms, slides, and releases under varying force. Such data are slow to collect through robot teleoperation, which often lacks force feedback and is constrained by gripper geometry. We ask whether these dynamics...

Yu-Xiang Ma, Wei-Han Xu, Peng-Fei Ye et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.