Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets Through Dynamic Dense Scattering Media

Abstract

ABSTRACT Imaging and tracking objects moving along random, unpredictable trajectories through dense scattering media remains an open challenge with direct applications in autonomous navigation, underwater robotics, and biomedical sensing. Here we present, to our knowledge, the first end‐to‐end brain‐inspired neuromorphic system that accomplishes both tasks simultaneously. A dynamic vision sensor (DVS) captures photons interacting with dynamic targets while suppressing the static “foggy” scattering background, yielding sparse spike trains. These feed directly into a deep spiking neural network (SNN) with dual modules: an Object Tracking Module for real‐time localization and an Object Reconstruction Module with residual refinement for high‐fidelity spatial recovery. Temporal memory in leaky integrate‐and‐fire neurons lets the network accumulate evidence across time steps, reconstructing consistent target geometry despite substantial motion‐induced variation in spike patterns. We validate the system in transmission and reflection geometries using solid optical phantoms, turbid water, and dynamic fog, with two‐dimensional targets of increasing complexity—MNIST digits, Kanji characters, and birds executing naturalistic trajectories. The system achieves structural similarity indices up to 0.96 and mean squared errors as low as 0.004, with full‐sequence inference completing in under 15 ms. Energy analysis indicates 18‐fold reductions relative to equivalent artificial neural networks, positioning the approach for emerging ultralow‐power neuromorphic hardware.