ViPoser: Sparse-IMU Based Human Pose Estimation with Distilled Vision Foundation Priors

Abstract

Human pose estimation using inertial measurement units (IMUs) has gained increasing attention due to its ability to enable camera-free motion capture for numerous human-centered applications. However, estimating full-body pose from only one to three daily carried commercial-off-the-shelf IMU devices (e.g., smartphones, smartwatches, and earphones) is highly under-constrained, as sparse sensor coverage and noisy signals often lead to unstable predictions and physically implausible body configurations. To address this challenge, we propose ViPoser, a lightweight pose estimation framework that transfers human-structure priors from a large vision foundation model into an IMU-based model through knowledge distillation. Specifically, we analyze the human-centric vision model Sapiens to identify layers rich in biomechanical and skeletal knowledge, distill these representations into a compact student Transformer, and integrate the distilled priors into a multi-stage pose estimation pipeline that performs local joint estimation, full-body masked completion, and global motion recovery. This design enables the model to infer anatomically plausible full-body poses even under sparse IMU observations while remaining suitable for real-time edge deployment. Experiments on multiple real-world datasets show that ViPoser consistently outperforms state-of-the-art IMU-based methods such as IMUPoser and MobilePoser, achieving 10~30% lower pose estimation error. In addition, ViPoser produces more anatomically realistic poses, significantly reducing failure cases observed in prior methods where predicted limbs intersect the body or violate biomechanical constraints.