The results support the effectiveness of combining LiDAR-camera fusion with instance-aware sparse temporal modeling and reduce the reliance on repeatedly propagating full-scene fused BEV features and concentrates temporal interaction on target-related instance states.
Abstract
With rising performance demands for autonomous driving perception systems, bird's-eye-view (BEV) perception that combines heterogeneous sensor data and temporal information has become a key research focus. Recent studies have advanced either multimodal temporal detection through BEV-level feature aggregation or sparse object-centric temporal modeling. However, integrating these two directions while preserving modality-specific temporal states remains less explored for LiDAR-camera detection. In this work, we present TempoCross, a 3D detection method based on instance-aware sparse representations for multimodal temporal fusion. TempoCross encodes features from different timestamps and sensor modalities in a unified instance space, enabling adaptive extraction of critical target information from modality- and time-specific states. Initially, both branches perform a preliminary cross-modal fusion to generate queries for the current frame. In the motion compensation module, a hybrid motion modeling strategy reduces alignment discrepancies between historical and current instances caused by complex object motion. This strategy combines explicit rigid-body transformations with implicit learnable deformation residuals, improving both accuracy and robustness in cross-frame instance association. Next, temporal-aware enhancement integrates the initial queries and current-frame features with motion-compensated historical instances. Finally, a lightweight cross-attention fuses current and historical instances from both branches. This formulation reduces the reliance on repeatedly propagating full-scene fused BEV features and concentrates temporal interaction on target-related instance states. On the nuScenes test set, TempoCross achieves 74.1% mAP and 75.7% NDS, outperforming mainstream baseline detectors. The results support the effectiveness of combining LiDAR-camera fusion with instance-aware sparse temporal modeling.
Camera-based 3D object detection has attracted widespread attention for autonomous driving applications. However, existing methods often lack effective feature screening mechanisms, resulting in an extremely low spatio-temporal signal-to-noise ratio in complex scenes. Specifically, distracting background projections, feature misalignment caused by dynamic objects, and frequent occlusions jointly lead to severe ambiguity and loss of object features. To alleviate these issues, we propose STRDet, a robust 3D object detection framework based on progressive spatio-temporal feature refinement. First, we propose a Semantic-guided Context Refinement (SCR) module that explicitly suppresses background interference prior to the view transformation, thereby blocking noise propagation. Second, we design the Differential-aware Feature Alignment (DFA) and Residual-based Adaptive Gated Fusion (RAGF) modules, which leverage feature difference maps as motion saliency indicators to guide deformable alignment and employ gating mechanisms to selectively integrate historical motion cues, effectively resolving dynamic alignment failures and mitigating feature loss under occlusion. Extensive experiments on the nuScenes dataset demonstrate that STRDet effectively enhances feature purity and coherence, yielding significant improvements and achieving 46.95\% mAP and 55.37\% nuScenes detection score.
Bing Li, Lie Guo, Longxin Guan et al.· Measurement science and tech...· 0 citations
Event-based cameras provide a powerful sensing modality for capturing dynamic scenes with high temporal resolution and low redundancy. However, leveraging modern deep learning architectures often benefits from transforming asynchronous event streams into suitable intermediate representations. The Compact Spatio-Temporal Representation (CSTR) addresses this by encoding events into a three-channel image-like format compatible with standard vision backbones, but its reliance on mean timestamps limits its ability to model long and complex temporal dynamics. In this study, we propose a Generalized Compact Spatio-Temporal Representation (gCSTR), a simple yet effective extension of the CSTR that enhances temporal expressiveness by constructing complementary representations in the spatio-temporal $xt$ and $yt$ projection planes. These representations preserve the fine-grained temporal structure while maintaining compatibility with conventional convolutional neural networks. To effectively combine multiple gCSTR representations, we introduce the Parallel Specialization Network (PaSNet), a multi-branch architecture. Each branch is trained on a distinct gCSTR view, allowing specialization to the structure of each representation. We evaluate gCSTR and PaSNet across a wide range of event-based object and action recognition benchmarks and demonstrate substantial improvements over the standard CSTR, particularly for long and complex action sequences. Our approach achieves state-of-the-art performance on several challenging benchmarks, while matching or closely approaching prior work on others, and provides new insights into when spatio-temporal projections are beneficial for event-based vision tasks.
Per Nyblom, David Gustafsson, Tomas Wilkinson· IEEE Access· 0 citations
The proposed recurrent cross-frame module aggregates historical context from the previous state to enhance the coarse ground feature of each current frame and facilitates satellite candidate-region classification and hierarchical fine-grained features enable precise local offset estimation.
Motion-only multiobject tracking (MOT) suffers from ID switches in uniform-appearance and deformation-heavy scenes. In these settings, appearance cues become less reliable, so stable identities depend mainly on motion information. Existing methods often process all bounding-box variables together, which can weaken cues needed for prediction and matching. We address this problem by treating each trajectory as a heterogeneous multivariate time series (MTS) and redesigning the motion-only pipeline for embedding, encoding, and matching. HSMTrack separates box-state variables before modeling their temporal and cross-variable relationships, then uses deformation-aware matching for identity association. The method requires no appearance branch and can serve as a post-detection motion module in vision-sensor tracking pipelines. Its SSM-based encoder has linear complexity with respect to trajectory length, reducing modeling cost compared with attention-based alternatives. HSMTrack achieves 59.6 IDF1 and 42.9 AssA on DanceTrack, and 77.9 IDF1 and 67.2 AssA on SportsMOT. Under a unified end-to-end protocol, it reaches 34.1 frames/s on RTX 4090 and 10.5 frames/s on Jetson Orin NX.
Tianjing Cheng, Qingyuan Yu, Bo Jiang et al.· IEEE Sensors Journal· 0 citations
Robust object detection for autonomous driving requires perception models that remain reliable when visible imagery is degraded by darkness, glare, rain, fog, motion blur, or long-range small targets. Visible and thermal infrared cameras provide complementary evidence, yet many RGB–thermal detectors fuse modalities, mainly as aligned tensors, and may underuse relational structure in channel responses, spatial layouts, semantic scales, and modality-specific uncertainty. This paper presents TopoGraph-Fusion, a hierarchical graph-guided dual-modal object detector that formulates fusion as topology-aware reasoning rather than direct feature concatenation. The proposed framework builds a dual-stream backbone for RGB and thermal images, constructs channel-wise topology through a channel-topology graph aggregation module, derives relation-aware spatial and channel global attention from affinity graphs, and replaces fixed feature-pyramid communication with a Graph-Guided Feature-Pyramid Network. A topology-regularized detection objective further encourages stable cross-modal correspondence while suppressing noisy all-to-all connections. Experiments on M3FD, FLIR, RGBTDronePerson, and VEDAI512 cover road scenes, adverse illumination, drone–person perception, and aerial vehicle detection. Within this validation scope, the results and visual analyses indicate that topology-guided fusion improves small-object recall, cross-modal consistency, and robustness under modality imbalance.
Pu Yu, Yanshan Ma, Yuheng Li et al.· Symmetry· 0 citations
Multi-modal object detection is essential for robust scene understanding in challenging conditions, including low-light and adverse environments. Recent vision foundation models (e.g., DINOv3) have exhibited strong representation capabilities, yet adapting them to multi-modal scenarios remains challenging. Existing dense cross-modal fusion strategies often force heterogeneous modalities to interact indiscriminately, which may introduce redundant information and disrupt the valuable pre-trained representations. To address this issue, we revisit multi-modal fusion from the perspective of socialized learning and propose adapter to DINOv3 (A2DINOv3), a multi-expert collaboration framework with a Socialized Collaboration Protocol (SCP). Specifically, RGB and infrared branches are modeled as heterogeneous experts that independently preserve their specialized knowledge while exchanging complementary information through selective and constrained interactions. This design mitigates harmful cross-modal interference and prevents degradation of pre-trained priors during adaptation. Furthermore, a zero-initialization strategy is introduced to gradually activate cross-modal collaboration, enabling a smooth transition from modality-specific learning to cooperative representation learning. Extensive experiments on four multi-modal benchmarks, including aerial detection (GAIIC), autonomous driving (FLIR), low-light surveillance (LLVIP), and diverse real-world scenarios (M3FD), demonstrate that A2DINOv3 consistently achieves state-of-the-art performance in multi-modal object detection.
Jiekang Feng, Zhihe Fan, Yunqi Zhu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
