Skip to content
Book Open access

Simulating a Dextrous Hand For Robotics With OpenUSD

Jul 2026 · Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Courses · 0 citations · 1 references

TL;DR

This hands-on course presents a workflow for configuring robot simulations using OpenUSD and PhysX in NVIDIA Isaac Sim, with an emphasis on physics tuning, stability, and asset authoring best practices, and situates OpenUSD’s composability within the broader physical AI ecosystem.

Abstract

Accurate, stable simulation is foundational to robot learning, digital twins, and physical AI. This hands-on course presents a workflow for configuring robot simulations using OpenUSD and PhysX in NVIDIA Isaac Sim, with an emphasis on physics tuning, stability, and asset authoring best practices. The course frames physics tuning as a prerequisite for meaningful training and evaluation, and situates OpenUSD’s composability within the broader physical AI ecosystem. Attendees work with a production Inspire underactuated robot hand asset, inspecting the scene—joints, masses, and collision shapes—to diagnose why the asset fails to simulate stably. Through guided activities, they configure colliders, select appropriate collision shapes, apply collision filters, and resolve overlapping-collider artifacts. They then configure joint drives and tune PD gains—stiffness and damping—using the Gain Tuner to achieve stable, responsive behavior. The session concludes with a demonstration of the tuned asset and an overview of downstream workflows in Isaac Lab. Outcomes include a repeatable workflow for preparing robot USDs for simulation, established best practices for colliders and joint drives, and hands-on experience with OpenUSD and PhysX in a production-grade toolchain.

Read PDF

Similar papers

Open access Jul 2026

Practical Integration of Open-Source Control Architectures on Custom Quadruped Robots: Simulation Validation and Hardware Interfacing

Custom-built quadrupedal platforms offer an accessible path for robotics research, yet researchers frequently encounter a “framework gap” when porting complex open-source control software to custom hardware. This paper documents the integration and validation process of Scotty, a custom quadrupedal robot, evaluating two distinct open-source control frameworks: the torque-based MIT Mini Cheetah and the position-based CHAMP architectures. Due to minimal documentation, hardware-dependent complexities, and a tightly coupled architecture, it was difficult to achieve a satisfactory result using the MIT framework within our system’s scope. Conversely, CHAMP’s structured integration documentation enabled the deployment of a locomotion pipeline validated in Gazebo simulation alongside a functional hardware interface middle layer. To overcome CHAMP’s lack of native operational state management, we developed a custom state-based controller with a web-based GUI that safely orchestrates transitions across Idle, Ready, Down, Stand, and Walk configurations. While walking was successfully validated in the simulation environment to verify the control software pipeline, physical hardware evaluation was restricted to individual joint control, localized leg movements, and GUI-based parameter tuning. Full closed-loop hardware locomotion was not achieved, as the extensive tuning of gait parameters and controller gains under full system weight was bounded by project constraints. The integration logs, practical hurdles, and architectural lessons documented in this work are shared openly to provide a clear, transferable roadmap for future developers of robotic systems.

Vishnudev Kurumbaparambil, Subashkumar Rajanayagam, Stefan Twieg · 0 citations
Preprint Jul 2026

DemoBridge: A Simulation-in-the-Loop Toolkit for Single-View Human Demonstration Retargeting

We present DemoBridge, an toolkit that turns a single-view RGB stereo recording of a human hand demonstration into an executable, physics-validated robot-arm trajectory. Retargeting across the embodiment gap is hard. A robot arm reaches a target with a long, articulated body whose links carry far more collision volume than a hand. Solving inverse kinematics for the mapped end-effector pose often yields no collision-free solution, and a trajectory imposes this at every waypoint. A single view adds noise, leaving the demonstrated reference inaccurate. At the core of DemoBridge is a single collision-aware planner. It optimizes the whole joint trajectory at once, reasoning jointly over alternative grasp poses, whole-arm and grasped-object collision, and fidelity to the demonstrated path. A physics simulator runs in the loop. It validates each phase as it is produced and backtracks on failure, so a demonstration that cannot be reproduced as given is re-planned rather than discarded. The resulting action sequence is dynamically stable and faithful to the demonstrated manipulation. It also doubles as a ready-to-use simulation rollout for policy learning. Grasp timing is inferred automatically, and the perception backends, robot, and pipeline stages are swappable from configuration. We evaluate whole-pipeline retargeting on three real-demonstration tasks and the planner on a controlled synthetic benchmark. Our code is available at https://gitlab.kuleuven.be/u0123974/demo-bridge/ .

Zehao Wang, Fabien Despinoy, Sergey Zakharov et al. · 0 citations
Preprint Aug 2026

DexMani: Human-Derived Manipulability Guidance for Dexterous Rotation

This work introduces DexMani, a framework that transfers human demonstrations as contact-conditioned manipulability evolution and uses it to guide downstream reinforcement learning, enabling rotation skills to be acquired across robot embodiments with distinct kinematics and active-contact configurations.

Xiaoyang Chen, Shengcheng Luo, Haoran Guo et al. · 0 citations
Preprint Jul 2026

Ace! Motion Planning of Professional-Level Table Tennis Serves with a Robot Arm

Table tennis, a dynamic, compact, and popular sport, has received significant attention as a robotics benchmark over the last decades. Most of the research has focused on the rally aspect - returning an incoming ball - requiring high-speed vision, agile motion planning, and tight closed-loop control. However, the other component of table tennis gameplay - the serve - is comparatively a quite unexplored research problem, that in fact requires pushing physics modeling and control to the extremes. Achieving competitive serves with a robot presents domain-specific challenges, such as high-spin generation from a spinless ball, precise aiming, or multi-objective optimization. In this work, we present a novel approach for generating official rule-compliant serves by combining motion primitives, Model Predictive Control, and Bayesian Optimization. Serves generated in this way offer a wide and controllable variation of spins of up to 550 rad/s, and speeds of up to 6.7 m/s, matching and even surpassing those of elite table tennis players.

G. Torrente, Guilherme J. Maeda, Divij Grover et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.