This paper analyzes two features used in event-based corner detection---the eigenvalues of the structure tensor and the spatiotemporal density values---and shows that they are motion cues, and hypothesize that these features, combined with local geometric information, can enhance motion estimation tasks.
Abstract
Event cameras capture intensity changes asynchronously with high temporal resolution, requiring novel preprocessing methods for downstream tasks. Unlike static intensity snapshots, event data inherently encode information about scene dynamics and object motion, meaning that features derived from events can exhibit behaviors with no direct analogue in frame-based vision. In this paper, we analyze two features used in event-based corner detection---the eigenvalues of the structure tensor and the spatiotemporal density values---and show that they are \emph{motion cues}. We hypothesize that these features, combined with local geometric information, can enhance motion estimation tasks. To validate this, we first theoretically analyze how the eigenvalues of the structure tensor at moving corner points relate to the direction of motion. We then design controlled experiments on a synthetic dataset, confirming that extending local geometric features with eigenvalues and density values provides complementary motion information and is robust to texture and shot noise. Finally, we integrate the proposed features into a state-of-the-art event-based optical flow network and evaluate on the real-world DSEC benchmark, where the added features consistently improve accuracy, with the largest gains in data-scarce scenarios and for lower-capacity models. The code for this paper can be found at: \href{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}.
Optical flow forms a fundamental information for various motion related vision problems: e.g., SLAM, visual odometry, and object motion estimation. Event cameras are ideal vision sensors for on-line, dynamic tasks that require optical flow estimation, as they have high temporal resolution, high dynamic range and low latency. However, efficiently decoding optical flow from events for high frequency operation while maintaining accuracy is still an open problem. Batch-based optical flow algorithms (CNN or contrast maximisation) accumulate event data over a short period of time and achieve state-of-the-art performance in terms of accuracy, but at the cost of algorithm latency and lower update rates (on par with traditional cameras). In contrast, event-by-event algorithms only compute flow vectors in small, local regions, achieving a lower latency, but losing accuracy when global information is ignored. In this paper, we introduce a spatio-temporal registration framework to increase accuracy of current state-of-the-art event-by-event flow estimation, while also introducing a twofold algorithm acceleration approach and a real-time implementation strategy to mitigate the impact of computation scaling with event rate. We evaluate our event by-event optical flow algorithm on MVSEC, achieving state-of-the-art results for event-by-event algorithms, and performance comparable to batch-based methods. Our method is also computationally efficient, enabling processing of the higher resolution DSEC dataset, and is the only event-by-event algorithm tested to run completely in real time. Furthermore, we demonstrate its effectiveness and efficiency through qualitative evaluations on the ECD and the high-resolution M3ED datasets. Finally, we introduce a moving object dataset, which is outside the autonomous driving domain, to evaluate the general applicability of the proposed optical flow algorithm. The code is available open-source [CODE AVAILABLE ON ACCEPTANCE].
Zhichao Li, Arren J. Glover, Lorenzo Natale et al.· IEEE Transactions on Pattern...· 0 citations
Event cameras capture pixel-level intensity changes with microsecond resolution to produce highly sparse asynchronous data streams. For visual perception in latency-critical environments, we propose a lightweight, training-free framework for discovery of moving objects based on spatio-temporal clustering. This framework is driven by two core contributions. First, a linear-time Spatio-temporal Probabilistic Event Filter (SPEF) that introduces an adaptive event acceptance threshold to distinguish salient motion structures from background noise. Second, an Event Morton Code Clustering (EMCC) module that bypasses expensive distance matrix computation to efficiently group events for unsupervised discovery of moving objects. On the E-MLB dataset benchmark, SPEF achieves the best denoising performance among classical filtering methods and remains competitive with learning-based approaches without requiring any offline training. On object discovery, EMCC achieves the highest overall accuracy and lowest execution time across the FRED and eTraM datasets, outperforming established density-based clustering baselines by a substantial margin. Overall, this work establishes a new performance benchmark for classical object discovery in event data, providing a highly scalable, training-free solution for resource-constrained visual perception. The code is available at https://github.com/PrathamShenwai/SPEF_EMCC
P. G. Shenwai, H. Singh, Sridhar Ravi· 0 citations
Motion-only multiobject tracking (MOT) suffers from ID switches in uniform-appearance and deformation-heavy scenes. In these settings, appearance cues become less reliable, so stable identities depend mainly on motion information. Existing methods often process all bounding-box variables together, which can weaken cues needed for prediction and matching. We address this problem by treating each trajectory as a heterogeneous multivariate time series (MTS) and redesigning the motion-only pipeline for embedding, encoding, and matching. HSMTrack separates box-state variables before modeling their temporal and cross-variable relationships, then uses deformation-aware matching for identity association. The method requires no appearance branch and can serve as a post-detection motion module in vision-sensor tracking pipelines. Its SSM-based encoder has linear complexity with respect to trajectory length, reducing modeling cost compared with attention-based alternatives. HSMTrack achieves 59.6 IDF1 and 42.9 AssA on DanceTrack, and 77.9 IDF1 and 67.2 AssA on SportsMOT. Under a unified end-to-end protocol, it reaches 34.1 frames/s on RTX 4090 and 10.5 frames/s on Jetson Orin NX.
Tianjing Cheng, Qingyuan Yu, Bo Jiang et al.· IEEE Sensors Journal· 0 citations
This paper proposes a semantics-guided scene decoupling module that separates Gaussian primitives into static and dynamic components based on motion vectors, and introduces a motion-aware densification module for motion compensation, which alleviates the incomplete rendering of dynamic objects caused by insufficient spatio-temporal information.
Chulin Zhao, Xue Wang, Guoqing Zhou et al.· IEEE Transactions on Visuali...· 0 citations
Robust 3D reconstruction is essential for robotics and embodied perception. Recent feed-forward approaches such as DUSt3R have demonstrated impressive progress in dense 3D reconstruction from RGB images, achieving global geometric consistency and strong generalization. However, extending such dense 3D reconstruction to event cameras remains challenging due to their asynchronous, sparse, and highly dynamic nature, as well as the lack of large-scale, well-labeled datasets. In this work, we introduce Event3R, a feed-forward framework that directly maps asynchronous event streams to globally consistent 3D point clouds. Event3R represents incoming events as spatial-temporal voxels, enabling time-aware feature integration through a temporal attention module that enhances the module's temporal feature learning. To further strengthen temporal representation learning and reduce reliance on labeled data, we propose a Masked Bin Modeling (MBM) strategy for self-supervised pre-training, enabling robust temporal representation learning with minimal labeled data, and retain it as an auxiliary fine-tuning objective. In addition, contrastive alignment and consistency regularization losses are incorporated during fine-tuning to reinforce structural correspondence and temporal coherence across views. Extensive experiments on both synthetic and real-world benchmarks demonstrate that Event3R achieves robust, temporally consistent, and globally aligned 3D reconstructions, significantly outperforming existing event-based methods.
This work proposes an adapter-based framework that incorporates event-derived cues into a pre-trained image-to-video diffusion model with minimal architectural changes and consistently outperforms existing state-of-the-art approaches.
Guixu Lin, Yuyang Yu, Xiang Ji et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
