Skip to content
Open access

OAHPE: occlusion-aware hybrid routing for efficient and robust 3D human pose estimation in mixed-occlusion videos.

Jul 2026 · Scientific Reports · 0 citations
Medicine

TL;DR

The OAHPE model is proposed to address the dual challenges of high uncertainty in occluded frames and high redundancy in visible frames during 3D human pose estimation from video, and attains superior performance with substantially lower parameter and memory costs.

Abstract

Attention-mechanism-based methods for video-based 3D human pose estimation (HPE) have demonstrated strong performance in spatio-temporal modeling. However, the quadratic complexity introduced by attention leads to substantial computational and memory overhead during long-sequence inference, thereby hindering lightweight deployment. Inspired by the long-range modeling capability of the state space model (SSM) with linear complexity, we propose the Occlusion-Aware Hybrid Pose Estimation (OAHPE) model to address the dual challenges of high uncertainty in occluded frames and high redundancy in visible frames during 3D human pose estimation from video. This model employs a differentiated dual-path modeling strategy. For occluded segments, we integrate a multi-scale dilated convolutions with linear-complexity-based Mamba module, and employ a dual local-enhancement mechanism to capture fine-grained joint structures and long-range inter-joint dependencies at the full frame rate. For visible segments, we design an online selection mechanism to reduce temporal redundancy, and introduce a distillation-based temporal recovery objective to enforce dynamic consistency. Finally, predictions from the two paths are backfilled, aligned, and fused at the original sequence resolution. Comprehensive quantitative and qualitative evaluations on the Human3.6M and MPI-INF-3DHP benchmarks demonstrate that OAHPE achieves mean per-joint position errors of 43.5 mm and 28.5 mm, respectively, while reducing the model size and training memory footprint to 2.6 M parameters and 12.5 GB. These results indicate that the proposed method attains superior performance with substantially lower parameter and memory costs, achieving a favorable trade-off between pose estimation accuracy and computational efficiency.

Read PDF

Similar papers

Conference Jul 2026

FG-Net: frequency-domain and Gaussian joint enhancement for 3D human pose estimation

Transformers have achieved remarkable performance in video-based 3D human pose estimation, yet their high computational cost hinders deployment on resource-constrained devices. To balance accuracy and efficiency, this paper proposes an efficient plug-and-play joint enhancement framework, FG-Net, which integrates the Frequency Enhancement Module (FEM) and Gaussian Enhancement Module (GEM) to boost the performance of video pose Transformers. On this basis, FEM calibrates the semantic consistency of multi-scale features through frequency-domain detail enhancement and deformable spatial alignment, compensating for information loss caused by sampling. GEM constructs graph attention based on human skeletal topology, combined with temporal Gaussian smoothing and residual fusion, to adaptively strengthen joint structures, suppress noise and temporal jitter. The two modules work in synergy, enabling the model to achieve efficient inference while being more robust to low-quality video inputs such as occlusion and motion blur. Experimental results on the public dataset Human3.6M demonstrate that the proposed method achieves 39.87 mm MPJPE, obtaining state-of-the-art accuracy with lower computational complexity. The framework is generic and can be seamlessly integrated into mainstream video pose Transformers, providing an effective solution for real-time 3D human pose estimation in resource-constrained scenarios.

Shaojie Cheng, Dazheng Zhou, Ran Wei et al. · 0 citations
Open access Jul 2026

SMG-YOLO: an efficient human pose estimation method

Human pose estimation is a fundamental task in computer vision that aims to localize key human joints in images. Although you only look once (YOLO)-based pose estimation methods provide advantages in computational efficiency and deployment convenience, they still face challenges in complex backgrounds, occlusions, small-scale keypoints, and structural inconsistency among predicted joints. To address these issues, this paper presents SMG-YOLO, an enhanced YOLO-based pose estimation framework built upon Hyper-YOLO-Pose. The proposed model integrates three pose-oriented components: the selective boundary aggregation (SBA) module, the mixed aggregation network (MANet)-StarC module, and a GroupNorm-based pose detection head, namely group normalization (GN)-Pose. The SBA module is adopted to strengthen semantic-spatial feature interaction between high-level semantic features and low-level spatial cues. The MANet-StarC module incorporates the StarC unit into the MANet structure and combines nonlinear feature modulation with context anchor attention to enhance contextual feature representation. GN-Pose introduces GN into the pose prediction head to improve keypoint regression stability. Experiments on the MPII human pose dataset show that SMG-YOLO achieves 85.30% AP50 and 47.70% AP50:95, improving the Hyper-YOLO-Pose baseline by 2.30 and 2.40 percentage points, respectively. Additional ablation experiments further verify the complementary effects of the proposed components. Moreover, model-forward speed testing on an RTX 3060 GPU shows that SMG-YOLO achieves 49.1 FPS, indicating practical inference efficiency under the tested setting. These results demonstrate that SMG-YOLO provides a favorable accuracy–complexity trade-off for human pose estimation.

Haiyun Zhou, Yicheng Yu, Ping Wu · 0 citations
Open access 2026

UnifiedPoseNet-A Lightweight Shared Pose Estimation Model for Human and Vehicles

Pose estimation is a key aspect of action recognition, study of behaviors and spatial relationships from video data. Traditional pose estimation methods are designed exclusively for keypoint detection of either humans or vehicles and incur high inference time and complexity. In this paper, we present UnifiedPoseNet, a lightweight convolutional network that performs 2D keypoint estimation simultaneously for both humans and vehicles in a single forward pass without depending on a bounding box. This joint approach addresses challenges in smart city applications like surveillance and traffic monitoring- where pedestrians and cars interact. The UnifiedPoseNet is built on a novel HVR-Net backbone that learns spatiotemporal features for both entities jointly, coupled with a Differentiable Feature Recombination Block (DFRB) to exchange contextual cues between human and vehicle representations. Dual decoders generate heatmap confidence maps for 17 human keypoints and 6 vehicle keypoints, enabling pose detection in mixed-scene environments. Our model employs a novel hybrid multi-task loss function combining focal and entity-aware terms: each training image carries an entity presence mask to enable focus only on relevant keypoints, thus making learning robust to missing annotations. Our model achieves a keypoint accuracy of 88.4% for humans and 85.9% for vehicles, with an FPS of 23 and a model size of 4.1MB demonstrating real-time edge efficiency. Comparative results against lightweight state-of-the-art baselines for human and vehicle pose estimation—demonstrate that UnifiedPoseNet attains comparable accuracy to specialized models while requiring significantly fewer parameters and offering a unified framework for simultaneous human–vehicle keypoint estimation.

Reenie Tanya, Balika. J. Chelliah · 0 citations
Conference 2026

SGAFormer: Skeleton-Graph and Agent-guided Transformer network for monocular video-based 3D human pose estimation

Monocular video-based 3D human pose estimation remains challenging due to depth ambiguity, noisy 2D observations, and complex spatio-temporal depend-encies. Existing Transformer-based methods can capture global relationships, but they often lack explicit skeletal topology constraints and may introduce re-dundant dense temporal interactions. To address these issues, this paper pro-poses a Skeleton-Graph and Agent-guided Transformer network (SGAFormer) for skeleton-aware and temporally coherent 3D human pose estimation from 2D pose sequences. In spatial modeling, the proposed Skeleton-constrained Adaptive Graph-order Attention Module (SAGA) introduces skeleton-constrained dynamic graph attention and multi-order graph propagation to cap-ture joint self-information, direct skeletal connections, and indirect structural dependencies. By adaptively fusing different graph-order branches, SAGA en-hances dynamic joint interaction modeling under physical skeleton constraints. In temporal modeling, the proposed Agent-token Global-local Temporal At-tention Module (AGTA) reorganizes dense frame-wise interactions through agent tokens, while incorporating temporal positional bias and depthwise sepa-rable temporal convolution to enhance global-local motion representation. Ex-periments on Human3.6M show that SGAFormer remains competitive under detected 2D keypoint inputs, achieves superior temporal consistency, and ob-tains strong upper-bound performance with ground-truth 2D keypoints, demon-strating the effectiveness of the proposed spatial-temporal modeling frame-work.

Min Li · 0 citations
Open access Sep 2026

DiMMPose: A Diffusion-Mamba Hybrid Framework with Multi-Prompt for Efficient and Robust 3D Human Pose Estimation

Monocular 3D Human Pose Estimation (3D HPE) typically adopts a two-stage approach: estimating 2D joint positions from images and then lifting them to 3D coordinates, effectively reducing dataset bias inherent in direct methods. However, current lifting techniques face two key challenges: many Transformer-based methods rely on attention-based or staged spatial–temporal modeling, which can limit efficient long-range frame-joint reasoning, while diffusion models support probabilistic modeling of pose uncertainty but remain sensitive to joint-coordinate noise. We propose DiMMPose, a diffusion-based framework enhanced by Mamba’s state-space model for robust and efficient 3D pose estimation. Its denoising process consists of two coordinated modules. The Spatiotemporal Mamba Block (STMB) serves as the core feature extraction module, employing internal Pose Mamba components with bidirectional state propagation and linear complexity to efficiently model long-range frame-joint dependencies. STMB further refines these features through Spatiotemporal Scan and Merge, which traverses the same skeleton tokens in complementary frame-joint orders and fuses the resulting representations. The Multi-Prompt Mamba Denoiser (MPMD) combines structured prompts encoded by LongCLIP with learnable prompt representations to provide anatomical and motion-related guidance during denoising. DiMMPose achieves an average MPJPE of 28.9 mm on Human3.6M under the DET setting, with action-specific errors of 21.2 mm for Walking and 22.0 mm for WalkTogether. It improves over FinePOSE by 3.0 mm, reduces inference latency by 57.1%, and achieves 23.0 mm MPJPE on MPI-INF-3DHP (N = 243).

Xu Li, Xuefeng Guan, Chang Liu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.