STCTrack: bidirectional Mamba with dynamic attention for spatiotemporal-contextual tracking

Abstract

Accurate modeling of object appearance in tracking requires a comprehensive understanding of contextual information. This understanding helps handle complex appearance variations that frequently occur in dynamic environments. Existing trackers often fail to adapt well to large appearance changes. They rely on static template update processes, which cannot sufficiently capture these variations. They also overlook the fact that different regions in the reference frame contribute unequally to object localization in the search frame. This leads to an incomplete capture of long-range contextual dependencies. We propose STCTrack, a hybrid architecture that combines attention-based encoders with Mamba-enhanced decoders to address these challenges and enable holistic spatiotemporal modeling. In the encoder, a sliding-window self-attention mechanism dynamically captures discriminative object features across frames. This eliminates the need for handcrafted components. In the decoder, improved bidirectional Mamba layers build a continuous state memory that implicitly encodes long-term trajectory evolution. A multi-head cross-attention mechanism then aligns historical context embeddings with current frame representations. This step reduces information fragmentation. Extensive experiments show that STCTrack achieves performance highly competitive with existing state-of-the-art trackers on multiple datasets. Notably, it attains an AUC of 71.2% on the UAV123 dataset, which demonstrates its superior tracking performance.