A three-stage pipeline that converts human motion-capture recordings into dexterous robot policies with no real-robot training data is built, which transfers one human dataset to four morphologically distinct robot hands, and executes four contact-rich bimanual tasks on physical hardware with zero real-robot training data.
Abstract
Learning dexterous manipulation from demonstrations is bottlenecked by data: the contact forces that determine whether a grasp succeeds are absent from every scalable source of human demonstrations. This paper builds on two observations. First, what survives the change from a human hand to a robot hand is the contact structure of a demonstration - which finger regions touch which object locations, and in what order - rather than its joint motion. Second, physical consistency need not be engineered per task: a single residual reinforcement learning (RL) policy, trained once across diverse demonstrations, can repair kinematic recordings into physically consistent, contact-annotated trajectories, and the same residual formulation restores dynamic feasibility after retargeting. These observations yield a three-stage pipeline that converts human motion-capture recordings into dexterous robot policies with no real-robot training data: physics refinement with a simulated MANO hand recovers contacts and forces, contact-anchored retargeting transfers the demonstrated contact structure through an objective independent of hand morphology, and residual policy learning adapts the result to robot actuation. The pipeline reconstructs 25,454 single-hand trajectories (success 7.3% ->59.3%) and 25 dual-hand tasks (16.0% ->62.4%) with one shared policy per setting, transfers one human dataset to four morphologically distinct robot hands (+62.4 pp), and executes four contact-rich bimanual tasks on physical hardware with zero real-robot training data.
Learning contact-rich dexterous manipulation from human demonstrations provides a scalable source of interaction data, yet transferring such skills to robots remains challenging due to unreliable human--robot correspondence. Existing human-to-robot transfer methods typically rely on visual appearance or motion similari...
Xing-Ting Li, Yi-Fan Han, Zi-Jian Lin et al.· 0 citations
Weave is presented, a unified framework for learning whole-body dexterous humanoid-object interaction from captured human demonstrations that converts captured human-object interactions into executable robot-object references through contact-aware retargeting and approach-motion completion.
Liu Cao, Xing-Ze Wu, Jing-Zhi Cui et al.· 2 citations
DemoMimic (Dexterous Motion Mimic), a policy that manipulates objects by focusing on their geometry local to the contact points to improve sim-to-real consistency, yielding a single real-world policy that transfers across objects of varying shape, scale, mass, and friction wherever local contact structure is preserved.
Glove-based motion capture is emerging as a scalable approach to collecting dexterous-hand demonstration data. However, due to the kinematic gap between the human and robot hand, the recorded human motions cannot be executed directly on the robot, especially for contact-rich tool-use tasks involving in-hand reorientati...
Han Yang, Yian Wang, Yun-Long Song et al.· arXiv.org· 0 citations
Transferring human demonstrations to dexterous robots remains challenging because differences in hand morphology and contact dynamics often cause retargeted motions to fail at producing the intended object behavior. We present \textbf{FoLD}, a framework for learning dexterous manipulation of articulated objects through...
Hao-Wei Shen, Ti-Ngai Li, Yu-Meng Liu et al.· arXiv.org· 0 citations
This work proposes an LfD method that explicitly uses what physical interactions take place where and when, and discusses how robustness, generalization, and adaptivity can be explicitly implemented, which is generally lacking in the LfD literature.
A. H. G. Overbeek, H. van der Kooij, M. Vlutters· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.