Masked pose embedding for view and occlusion invariance in 2D pose analysis

Abstract

Pose embeddings provide a practical way to improve viewpoint invariance and occlusion robustness in 2D pose analysis by transforming sparse landmark coordinates into representations that preserve underlying 3D pose semantics. Existing approaches often rely on contrastive or variational objectives, which require careful pair construction, large batches, or additional training constraints. In this paper, we propose masked pose embedding (MPE), a transformer-based autoencoder trained with a single masked-reconstruction objective tailored to pose landmarks. MPE combines two key components: (1) view-angle canonicalization, in which randomly rotated 3D landmarks are projected to 2D and decoded toward the original canonical pose, and (2) landmark-level masking with masked-only supervision to improve robustness to partial visibility. We evaluate MPE in two complementary settings. First, on SARA and Human3.6M, we analyze whether distances in the learned embedding space correlate with ground-truth 3D pose distances under both controlled perturbations and under natural, unperturbed data. Second, on Yoga-82, we assess real image-based pose retrieval using YOLOv8x-pose landmarks under multiple synthetic occlusion patterns. The results show that MPE more faithfully preserves 3D pose semantics than recent single-frame and transformer-based representation baselines and achieves stronger retrieval accuracy among the compared methods under both full and partial landmark observations. These findings suggest that MPE provides an effective and training-stable solution for view- and occlusion-robust 2D pose embedding, while remaining highly parameter-efficient compared with recent single-frame and temporal representation baselines.