Skip to content
Preprint

ViewStruct: View-Grounded Articulation and Connectivity Inference

Oct 2026 · 0 citations · 34 references
Computer Science

TL;DR

ViewStruct is introduced, a framework for jointly predicting connectivity and articulation in objects and scenes and its view-grounded representation links predictions to observed regions and expresses joint geometry in a gravity-aligned, view-relative frame, without requiring category-specific canonical orientations.

Abstract

Interacting with unfamiliar objects requires understanding how their parts connect and move. We introduce ViewStruct, a framework for jointly predicting connectivity and articulation in objects and scenes. Given an RGB image with aligned part segmentation and depth, optionally supplemented by images or video, ViewStruct predicts rigid-body groupings, attachments, and joint parameters. Its view-grounded representation links predictions to observed regions and expresses joint geometry in a gravity-aligned, view-relative frame, without requiring category-specific canonical orientations. A pretrained vision--language model is adapted to predict structure in seconds, without requiring observed articulation motion or per-instance optimisation. Experiments demonstrate improved structural prediction over evaluated baselines and transfer to unseen categories, asset sources, and real-world human-interaction videos, despite task-specific training exclusively on synthetic data. Simulated manipulation experiments further demonstrate the utility of predicted joints for generating robot motion trajectories.

View source

Similar papers

Preprint Oct 2026

Relational Spatial Alignment for Vision-Language-Action Models

This work proposes a representation-alignment method that enables the injection of 3D spatial awareness without compromising the model's original expressivity, and effectively incorporates 3D spatial representations into VLA reasoning, achieving robust performance and enhanced spatial awareness across diverse manipulat...

Chao-Ran Zhu, Hengyi Wang, Cheng-Lin Cui et al. · 0 citations
Preprint Oct 2026

Relational Geometry Forcing for World Action Models

World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D p...

Yuan Xu, Yi-Xiang Chen, Qi-Sen Ma et al. · 1 citation
Preprint Oct 2026

Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling

Embodied world models should not only imagine plausible futures, but also perceive the geometric, semantic, and motion structure of the evolving world. We present GenSight, a unified embodied world model that brings future video generation and dense perception into a single pretrained video diffusion transformer. GenSi...

Hai-Tang Feng, Zhuo-Man Liu, Zhen-Nan Wang et al. · 0 citations
Sep 2026

MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence

MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...

Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al. · 1 citation

TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models

TRWORLDBENCH is introduced, a benchmark for evaluating embodied world models through synchronized head, left-wrist, and right-wrist videos and uses 19 metrics to assess tri-view consistency, task alignment, physical and 3D coherence, motion quality, temporal consistency, and visual quality.

Xuan-Yi Liu, Hao-Feng Wang, Rui-Qi Li et al. · 2 citations · ⚡1
Sep 2026

MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation

World-Action Models (WAMs) couple visual dynamics with action prediction, bringing the rich priors of pretrained video models to robotic manipulation. However, their multi-view interfaces typically tile images or concatenate tokens, leaving the geometric relationships among synchronized cameras implicit. This makes it...

Wen-Bo Chen, Tian-Fu Li, Hao-Xuan Xu et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.