ReBi: Bidirectional State-Space Rectification for Multimodal Micro-Action Recognition

Abstract

Multimodal micro-action recognition requires distinguishing subtle behavioral variations from limited visual evidence. Appearance and human-structure cues provide complementary information, yet conventional interaction mechanisms primarily emphasize cross-modal correspondence modeling while lacking explicit post-interaction correction, leaving spurious correlations and representation deviations insufficiently constrained before classification. To address this limitation, we propose ReBi, a bidirectional state-space rectification network that refines multimodal representations after cross-attention. ReBi adapts Mamba's selective state propagation into a bidirectional cross-modal process that reinforces informative interactions and suppresses unreliable ones. The resulting corrections are incorporated through zero-initialized modality-specific residual modulation, preserving the initial interaction mapping while learning asymmetric adjustments for each modality. We further introduce Bidirectional Hierarchical Consistency Calibration (HCC) to exploit the dependency between fine-grained Action categories and their parent Body categories. HCC uses Body predictions to support compatible Actions and aggregates related Action evidence to refine Body decisions, thereby reducing cross-level inconsistency. On MA-52, ReBi yields consistent Action and Body F1macro gains, reaching 67.47% Action Top-1 and 70.31% F1mean. These results demonstrate the effectiveness of selective cross-modal rectification and bidirectional hierarchical calibration for fine-grained micro-action recognition.