RoPT: Robust 6DoF Object Pose Tracking in Video

Abstract

Accurate 6-DoF object pose tracking is essential for robotic interaction, yet conventional monocular pipelines frequently suffer from catastrophic divergence under rapid object motion. We present RoPT, a robust tracking framework engineered to overcome this limitation by unifying visual perception, temporal kinematics, and global re-localization into a tightly coupled closed loop. Unlike existing open-loop trackers, RoPT delivers three critical capabilities that fundamentally alter system resilience. First, the visual perception component tracks object masks to robustly handle partial and severe occlusions. Second, a kinematic factor graph regularizes frame-to-frame updates, forecasts forward pose predictions, and drives a parallel multi-hypothesis verification scheme to bypass local minima. Finally, an active re-localization module transforms the pipeline into a closed-loop system, making it failure-aware and capable of deterministic self-recovery. We demonstrate the effectiveness of our framework across challenging benchmarks alongside real-world robotic demonstrations, achieving significantly higher tracking survivability under rapid motion.