Skip to content
Preprint

Perceptive Humanoid Motion from Simulation

Nov 2026 · 0 citations · 66 references
Computer Science

TL;DR

HALO is presented, a humanoid autonomy system powered by a pipeline that synthesizes vision-language-motion data across a thousand procedurally generated homes and orchestrates these skills to solve composite, long-horizon tasks.

Abstract

Humanoid robots have the potential to automate a wide range of physical tasks. Progress toward generalist humanoid autonomy, however, remains constrained by a data bottleneck: mapping pixels and free-form language to low-level joint commands requires paired supervision, and no corpus of such data exists at sufficiently large scale. Some approaches rely on human teleoperation and wearable capture devices, which require at least an hour of human time to produce an hour of data. We present HALO, a humanoid autonomy system powered by a pipeline that synthesizes vision-language-motion data across a thousand procedurally generated homes. The corpus scales with compute rather than human labor. HALO acquires a repertoire of physical skills, including sitting on chairs, navigating to named objects, stepping sideways through half-open doors, crouching and crawling beneath tables, and pushing boxes. It orchestrates these skills to solve composite, long-horizon tasks. Without ever seeing a real home during training, HALO transfers zero-shot to the physical world, achieving a 91% success rate across 360 trials in six unmapped real-world environments. See https://apple-aiml-research.github.io/ml-halo/ for code and videos.

View source

Similar papers

#machine learning Preprint Oct 2026

Workhorse: Learning Robust Whole-Body Humanoid Loco-Manipulation from Human Data

Humanoid robots still struggle to plan contact-rich whole-body manipulation from egocentric RGB and proprioception. Workhorse learns such manipulation from robot-free human demonstrations. A visual planner predicts five-link targets: the poses of the torso, both wrists, and both feet. A reinforcement-learning whole-bod...

Song-Bo Hu, Qia-Yuan Liao, Yu-Feng Chi et al. · 1 citation
Sep 2026

GAE: General Action Expert for Real-Time Humanoid Teleoperation

Humanoid avatars extend human physical presence beyond the body, enabling people to participate in social, service, and labor activities through remotely operated robots. This requires teleoperation systems capable of realizing diverse and dynamic whole-body behaviors while maintaining responsive human-robot synchroniz...

Yue-Fan Wang, Huai-Cheng Zhou, Xiao He et al. · 0 citations
Sep 2026

EgoAlign: Bridging the Human-Humanoid Gap for Long-Range Loco-Manipulation

EgoAlign, a data-construction framework that converts these demonstrations into action and state supervision compatible with a general-purpose, continuous whole-body controller, without collecting physical-robot demonstrations, is presented.

Yi-Ming Jiang, Jin Chen, Chong-Yang Xu et al. · 1 citation
#machine learning Preprint Oct 2026

VioLA: Learning Generalist Humanoid Control Policies from Human Data

Teaching a humanoid to follow instructions with its whole body runs into two obstacles. Its action space is large and tightly coupled: legs, arms, and fingers must move together while the robot keeps its balance, which makes joint-level actions hard to learn. And humanoid demonstrations are scarce, so current humanoid...

Mert Albaba, Jens Beißwenger, Anna Manasyan et al. · 0 citations
Preprint Nov 2026

HITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness

Enabling vision-language models (VLMs) to control humanoid robots requires both reliable whole-body interfaces and knowledge of how to use the body that carries over across tasks. We present HITCH (Harness Interfaces To Command Humanoids), a harness in which perception returns the 3D position of any point the VLM selec...

Qing-Zhou Lu, Chen-Xi Liu, Xin-Ming Wang et al. · 0 citations
Preprint Oct 2026

HuGe: Scaling Humanoid Generalist Vision-Language-Action Models with Egocentric Experience

Extending Vision-Language-Action models (VLAs) to humanoid robots requires coordinated whole-body control and diverse demonstrations that are costly to collect in real-world scenarios. Although egocentric human demonstrations provide a complementary source of task experience, differences in visual embodiment and missin...

Yang Liu, Jia-Cheng Liu, Heng-Tao Li et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.