Edge-AI-based Hybrid ANN–SNN for Real-Time Emergency Acoustic Event Detection in Automotive Systems

Abstract

To see and comprehend their surroundings, autonomous and sophisticated advance driver assistance systems require a very large count of cameras and LiDAR sensors in the automotive systems. However, due to some of situations like bad weather, poor lighting, and obstructions that can interfere with its dependability, leading to safety risks and perceptual malfunctions which lead to a risky and deadly outcome. Emergency sirens and other acoustic cues usually precede visual signals and continue to be perceived even in the event that these vision sensors fail or malfunction. Despite this benefit, due to the energy usage of the battery-operated automotives and their computational limitations, audio-based perception is rarely used. The hybrid Artificial Neural Network–Spiking Neural Network (ANN–SNN) framework developed in this study enables the identification of emergency acoustic incidents in cars in real time environments. Utilizing spectral traits extracted from actual audio data, a lightweight ANN is trained. Poisson, temporal, and delta encoding techniques are then used to transform these extracted spectral characteristics into spike representations. In order to provide rapid and energy-efficient inference, leaky integrate-and-fire neurons are used to process the spiking signals. The suggested system greatly decreases latency and the energy consumption while achieving accuracy on par with ANN baselines, according to experimental results. A browser-based real-time implementation further supports the practicality of this suggested approach for in-vehicle deployment.