MATHDance: Mamba-Transformer Architecture with Uniform Tokenization for High-Quality 3D Dance Generation

Abstract

Music-to-dance generation is a challenging and impactful task at the intersection of choreography, virtual reality, and generative content. However, traditional methods suffer from limited choreographic consistency. This oversight is reflected in unnatural limb movements, repetitive gestures, or overly stationary movements in the generated dances. To address these issues, we propose MATHDance, a novel architecture for music-driven 3D dance generation. By decoupling choreographic consistency into physical plausibility and aesthetic quality, MATHDance achieves significant performance improvements. It consists of two stages: (1) The High-Fidelity Dance Tokenization Stage (HFDT) pioneers the utilization of Finite Scalar Quantization (FSQ) for uniform tokenization. (2) The Hybrid Music-to-Dance Generation Stage (HMDG) employs a Mamba-Transformer hybrid architecture to autoregressively map music to dance tokens. Simultaneously, current evaluation metrics are based on handcrafted low-level features. Such limitations fail to capture the artistic essence of music and dance, thereby misleading the generative process toward producing dances that lack choreographic consistency. To address this, we introduce a retrieval-based evaluation protocol, which leverages the intrinsic connection between retrieval and generation tasks and utilizes contrastive learning to encode cross-modal music-dance semantics. Extensive experiments on FineDance and AIST++ demonstrate that MATHDance achieves state-of-the-art performance in both generation quality and computational efficiency, while validating the reliability of our evaluation protocol.