TAC-LOCO is proposed, a tactile-augmented unified reinforcement learning framework that encodes tactile array observations from compliant grippers into a compact latent representation and joins it with proprioception for unified control of the legs, arm, and gripper.
Abstract
Dynamic loco-manipulation requires legged robots to coordinate whole-body motion while maintaining stable physical interaction with grasped objects under uncertain external forces. While tactile sensing has been widely studied for robotic manipulation, its role in dynamic whole-body control remains largely unexplored. Existing works without tactile feedback commonly grasp firmly rather than regulate the grasp according to the interaction. We propose TAC-LOCO, a tactile-augmented unified reinforcement learning framework that encodes tactile array observations from compliant grippers into a compact latent representation and joins it with proprioception for unified control of the legs, arm, and gripper. With effective grasp stability reward design, the policy learns to simultaneously track body velocity and end-effector trajectories, moderate grasp force, and prevent object slip under both gradual load changes and sudden release events. We deploy the policy zero-shot on a Unitree Go2 with an Interbotix WidowX 250 arm and tactile gripper, demonstrating dynamic tactile-informed loco-manipulation under varying external interactions, achieving a 47% reduction in grasping force and an object drop rate of less than 1%.
In unstructured environments, endowing robots with the ability to dexterously and safely grasp unknown objects presents a critical challenge. Existing control methods struggle to adapt dynamically like human hands, failing to balance grasping stability and object safety. Inspired by human grasping mechanisms, we propose a grasping state regulation strategy based on visual feedforward and tactile gating reflexes to dynamically adjust grasping force. First, guided by the idea that visual information can provide object-dependent expectations before contact, we developed a Two-Stage Mass Estimation Framework and a Vision-Based Friction Coefficient Estimation Framework. They extract the object’s mass and friction coefficients as physical priors from visual inputs, providing reliable initial expectations for subsequent grasping force regulation. Next, the Multimodal State Classifier compares the expected tactile-state representation derived from physical priors and global visual information with the actual tactile-state representation extracted from real-time tactile feedback, and outputs discrete corrective actions. To evaluate performance, we introduce a new metric called the anthropomorphic rate. It quantifies the similarity between the robot-applied force and the human instinctive grasping force. We verified our framework through comprehensive offline and online experiments. Results demonstrate that our system achieves a 91.82% grasp success rate and a 92.65% anthropomorphic rate. These results demonstrate the effectiveness of the proposed strategy in real-world physical interactions.
Yu-Yao Qi, Tian-Le Wang, Yi-Da Fang et al.· IEEE Robotics and Automation...· 0 citations
Quantitative experiments showed that the proposed method generally outperformed the baselines in mass and CoM variations, particularly in terms of success rate, while maintaining robust performance across the evaluated conditions.
Jinseok Kim, Iksu Choi, Hunjo Lee et al.· Intelligent Service Robotics· 0 citations
This work proposes a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement and introduces an efficient visuo-tactile representation that tightly fuses object geometry with tactile feedback by associating tactile signals with finger identities.
Xirui Liang, Jiaqi Liang, Jing-Kai Xu et al.· 0 citations
Robotic grippers face substantial challenges in grasping and manipulating thin objects. Most existing grippers rely on highly precise approach and grasp motions, which limits robustness and reduces applicability. This paper explores thin-object grasping using books as a representative example. Here, we propose a novel solution that integrates an active surface with underactuated compliance to achieve stable grasping of thin objects without complex control. First, an underactuated gripper with an active surface is designed. The active-surface thumb performs in-hand repositioning of the target book without requiring adjustments of the robot arm or the other fingers, while the underactuated fingers establish compliant contact conditions with the environment, and the reconfigurable structure enables reliable grasping of books under different configurations. Second, we establish a kinematic model of the gripper, and determine the initial grasp postures for two representative scenarios (books lying flat on a desktop and books vertically packed in a shelf). Third, by analyzing the physical model of a book lying on a table and its interaction with the gripper and the environment, we systematically optimize the structural parameters and grasping strategy. Finally, extensive experiments validate the effectiveness of the proposed gripper and strategy. The results demonstrate strong robustness and adaptability when grasping thin objects placed flat (including books, paper, fabric, plastic film, and mouse pad), as well as a high success rate when grasping vertically packed books. Moreover, the proposed gripper can reliably complete long sequential"grasp-place"tasks.
In-hand manipulation allows multi-fingered dexterous hands to reconfigure grasped objects without releasing and regrasping them. This improves manipulation efficiency by reducing repeated grasp acquisition and large arm motions. However, most learning-based methods focus on reorientation, continuous rotation, or translation, whereas many tasks require joint control of object position and orientation. We formulate this capability as in-hand 6D object pose reaching: starting from an existing grasp, coordinated finger motions move the object to a palm-relative target pose. We present POISE (Palm-relative Object reaching In SE(3)), a sim-to-real reinforcement learning framework for this task. POISE combines diverse stable-grasp initialization, goal- and geometry-conditioned control, an adaptive 6D goal curriculum, and a compact reward scheme for pose reaching and grasp preservation. In simulation, diverse initialization raises held-out-grasp success from 40.1% to 51.5% and post-drop recovery from 33.8% to 72.9%; the curriculum raises full-range success from 6.2% to 59.5%. On hardware, the grasp-maintenance reward improves three-target sequence success from 20% to 80%. In real-world experiments, POISE reaches successive 6D targets without manual reset across multiple object geometries and wrist orientations, and recovers from external disturbances. To support further research in dexterous manipulation, we will release our code at https://junxiaolin.github.io/poise-website/.
Jun-Xiao Lin, Tian-Yue Wu, Jie Yin et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.