CoSafe: A Cooperative V2V Perception Framework with LLM Reasoning for Hazard Detection on Real Dashcam Data
TL;DR
The Cooperation Gain metric is introduced to quantify the proportion of cases in which hazard detection depends on inter-vehicle information sharing, and it is shown that on a real-world occluded-pedestrian scenario, cooperation provides an advance warning of at least 1.73 s before the following vehicle’s own detector registers the pedestrian.
Abstract
Cooperative perception through vehicle-to-vehicle (V2V) communication can resolve occlusions that single-vehicle systems cannot overcome, yet existing frameworks rely on simulated environments and expensive multi-sensor platforms. This paper presents CoSafe, a cooperative perception and reasoning framework built on real-world data acquired from dashboard cameras with integrated GPS. CoSafe extends a previously validated image-based positioning algorithm by adding YOLOv8 object detection, cooperative state fusion, Chain-of-Thought reasoning using a Large Language Model, and a deterministic rule-based safety validation layer. The core contribution is a spatial–temporal vehicle matching framework that associates frames captured from overlapping geographic locations at different timestamps, enabling cooperative hazard reasoning across asynchronous and partially observable vehicle streams. This paper also introduces the Cooperation Gain metric to quantify the proportion of cases in which hazard detection depends on inter-vehicle information sharing. On a real-world occluded-pedestrian scenario, cooperation provides an advance warning of at least 1.73 s before the following vehicle’s own detector registers the pedestrian, with a Cooperation Gain of 0.533 measured against manual human annotation. An ablation across single-vehicle, rule-only, and LLM-only configurations isolates the contribution of each component, and a negative-scenario test yields zero false alarms in normal traffic.