AI-based Missing Person Identification using YOLO and Deep Facial Embeddings

Abstract

Missing person identification using surveillance imagery remains a challenging problem due to adverse visual conditions, limited availability of reference images, and strict privacy constraints surrounding real-world data. This paper presents a hybrid deep-learning framework for missing person identification that combines YOLO-based person detection with deep face embedding models, specifically FaceNet and ArcFace. To address ethical and privacy limitations, a manually curated composite synthetic dataset is constructed by combining publicly available crowd and in-the-wild face datasets with additional curated images to realistically emulate CCTV conditions, including low illumination, occlusions, visually similar individuals, accessories, and clothing-matched decoys, while restricting each identity to only three to four reference images. The proposed system is evaluated across five YOLO variants (v8–v12) and a wide range of cosine similarity thresholds to analyze detection sensitivity, false positive behavior, and overall identification robustness. Extensive experiments conducted on 780 group images demonstrate that the YOLO–ArcFace pipeline achieves superior performance, reaching a peak identification accuracy of 97.50% with near-zero false positives, while the YOLO–FaceNet pipeline attains a maximum accuracy of 95.42% at an optimized operating threshold but exhibits higher susceptibility to false matches. Threshold–accuracy analysis and operating point comparison reveal that ArcFace embeddings provide stronger inter-class separation and greater stability under surveillance-specific distortions. The results highlight the importance of discriminative embedding models in safety-critical identification tasks and demonstrate that reliable missing person detection is achievable even under constrained and privacy-preserving settings, with future extensions proposed toward continuous video-based tracking using multi-object tracking algorithms.