Embodied Camera observations of Human Object carrying is presented, a large-scale synthetic dataset that pairs dense RGB-D scans of indoor scenes with recordings of an embodied human carrying everyday objects to context-appropriate destinations and shows that no single input modality is sufficient, highlighting the nee...
Xue-Fei Sun, Lorin Achey, K. Hamilton et al.· 0 citations
Reasoning over language instructions in embodied tasks such as robotics often requires understanding spatial relations from a speaker's situated perspective. Humans infer such perspectives from shared environmental knowledge, activity context, and commonsense. Recent vision-language models (VLMs) appear capable of spat...
Mimo Shirasaka, Haochen Zhang, Yonatan Bisk· 0 citations
SALT is introduced, a Semantically ALigned action Tokenizer that augments a VQ-VAE-style tokenizer with an auxiliary objective requiring a frozen vision-language model to recover the episode instruction from quantized action latents to substantially improve language-conditioned control.
Wen-Jie Li, Yash Jangir, Ignacy Stepka et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.