High-Fidelity Human Pose Transfer: A Unified Framework With Hierarchical Semantic Alignment and Gated Residual Fusion
TL;DR
A unified generative adversarial network (GAN) framework that integrates three novel, complementary mechanisms that generates high-fidelity, structurally faithful results from a single reference image, offering a robust solution for applications in virtual try-on, animation, and human image synthesis under challenging pose transformations.
Abstract
Pose transfer, a core task in human-centric image generation, aims to synthesise photorealistic images of a subject in novel poses while preserving identity and intricate clothing details. Existing methods, particularly under large pose variations such as extreme articulation or self-occlusion, often struggle with preserving fine-grained textures and maintaining structural consistency, leading to artifacts like distorted limbs and lost details. To address these challenges, we introduce a unified generative adversarial network (GAN) framework that integrates three novel, complementary mechanisms. First, a Hierarchical Semantic Aligner (HSA) establishes multi-scale semantic correspondence between source appearance and target pose features through local attention and global gating. Second, a Pose-Aware Feature Injection (PAFI) module explicitly models source-target pose discrepancy to generate dynamic modulation parameters for adaptive feature adjustment during decoding. Third, a Gated Residual Fusion (GRF) strategy adaptively balances local detail and global structural information via a learnable dual-branch gating mechanism. Evaluated on the DeepFashion dataset, our framework demonstrates significant improvements, achieving a 13.9% reduction in Fréchet Inception Distance (FID) compared to the MAGPT method, alongside superior scores in Structural Similarity Index (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS). Ablation studies confirm the individual contributions of each component. The proposed approach generates high-fidelity, structurally faithful results from a single reference image, offering a robust solution for applications in virtual try-on, animation, and human image synthesis under challenging pose transformations.