Smart Crosswalk Control Using OWL-ViT-Based Pedestrian Detection
Abstract
Pedestrian safety at urban crosswalks requires adaptive signal control systems capable of operating reliably across varying environmental conditions. Traditional vision-based solutions relying on closed-set object detectors often suffer from brittle performance when deployed under uncalibrated or adverse weather scenarios. To address these limitations, this paper presents a smart crosswalk control system based on the OWL-ViT vision-language model for zero-shot pedestrian detection and adaptive traffic signal management. The proposed framework integrates open-vocabulary text-prompt conditioning, a dynamic 40% Region of Interest (ROI) spatial filter to eliminate non-crossing background targets, and a rule-based signal switching logic. Experimental evaluation on an expanded traffic dataset of 140 images demonstrates that OWL-ViT effectively detects pedestrians across challenging environments—including rain, fog, and low-light scenarios—without task-specific retraining. The raw pedestrian detection yields a Mean Squared Error (MSE) of 2.2429 and a Root Mean Squared Error (RMSE) of 1.4976 against full-frame ground truth, underscoring the critical necessity of the downstream ROI filtering stage to isolate active crosswalk users. In traffic signal state decision-making, the system achieves a Precision of 1.0000, a Recall of 0.9664, and an F1-score of 0.9829. On the evaluated hardware, the system achieves an average execution latency of 1.9254 seconds per frame (0.52 FPS). The findings confirm the viability of vision-language models for intelligent transportation systems and outline future optimization strategies, such as model quantization and weather-aware prompt engineering, to achieve real-time edge deployment.