Skip to content
Preprint

MEgoVista: Multi-view Ego-aware Motion Estimation for Metric 4D Hands and Head in the Wild

Sep 2026 · 0 citations · 36 references
Computer Science

TL;DR

MEgoVista is offered as a measured route from egocentric video to metric hand supervision, one that widens where such labels can be gathered, one that widens where such labels can be gathered.

Abstract

Learning manipulation from human video requires high-fidelity hand-motion reconstruction in metric units. Today's metric hand labels come from studio rigs and instrumented headsets, and both are confined in the same two ways: neither leaves a prepared setting, and neither is checked against an independent reference. Unconstrained head-worn recording promises the opposite trade-off, scaling with the number of people wearing a device. We therefore introduce MEgoVista, an offline pipeline that turns a single unprepared MEgo View recording into metric two-hand and head motion in one gravity-aligned world frame. Three properties set it apart from existing egocentric reconstruction systems: first, it reconstructs in settings studio volumes and tabletop rigs cannot reach, settling hand ownership at detection so bystander hands stay out of the wearer's trajectory; second, it takes its metric gauge from calibrated stereo rather than a monocular prior, installing scale at initialisation so policies receive physical units, not arbitrary coordinates; third, both outputs are scored inside a motion-capture volume against independent Chingmu optical capture, under a protocol that audits its own reference and charges what a method declines to predict. MEgoVista is offered as a measured route from egocentric video to metric hand supervision, one that widens where such labels can be gathered.

View source

Similar papers

ESTHER: Egocentric Stereo Hand Estimation and Reconstruction in the Wild

ESTHER is proposed, a model whose stereo geometry, temporal reasoning, and output representation are designed for wearable egocentric stereo, and shows state-of-the-art accu?racy, superior external generalization, and robustness to the missing views, dropped frames, and lighting and motion blur extremes of real egocen...

Hong-Yu Ma, Hai-Rong Qu, Shi-Qi Zhao et al. · 0 citations
Sep 2026

InfiniHand: Streaming World-Space Hand Motion Estimation from Egocentric Video

InfiniHand is an end-to-end streaming feed-forward framework that jointly estimates MANO parameters, camera trajectories, and hand locations directly from uncalibrated egocentric video, achieving a 21% reduction in ARCTIC PA-p compared to ViDiHand while substantially mitigating world-space drift.

Ke-Rui Ren, Kai-Wen Song, Weiguang Zhao et al. · 0 citations
Preprint Sep 2026

Seeing the World and the Self from Egocentric Video

Experiments show that RESELF outperforms state-of-the-art methods designed for the individual tasks across depth estimation, camera tracking, and full-body motion estimation.

Kai Guan, Minchao Jiang, Ruichen WangLi et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Beyond Visual Quality: Evaluating Physical Consistency under Ego-Motion with EgoGenEval

EgoGenEval introduces EgoGenEval, a geometry-grounded, pose-free benchmark designed to evaluate the physical consistency of visual generators under ego-motion, and shows that pairwise supervision does not reliably improve camera-motion grounding and scene-state preservation together.

Yi-Lin Long, Chen-Ming Zhu, Zi-Tang Gou et al. · 0 citations
Preprint Aug 2026

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

This work introduces HumanTracker, a preference-aligned metric trained on 12K motion pairs containing 24K motions that better predicts human preferences and reveals contact and stability failures that kinematic metrics often miss.

Dai-En Liu, Ze-Kun Qi, Jia-Yu Zeng et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.