SEPIV-IDS: A Structured Evaluation Pipeline for In-Vehicle Intrusion Detection Systems

Abstract

Critical safety functions in modern vehicles rely heavily on intra-vehicle networks (IVNs), primarily via the Controller Area Network (CAN) protocol. The inherent vulnerabilities of CAN require robust intrusion detection systems (IDS) to mitigate adversarial threats. However, state-of-the-art IDS, especially AI-based approaches, often lack a comprehensive, well-defined performance analysis method. This work proposes and evaluates a structured pipeline for in-vehicle IDS, analyzing an autoencoder semi-supervised IDS as a practical case study. The method is validated on publicly available datasets, covering multiple attack types, with additional analysis of generalization capabilities. Performance is rigorously assessed using precision, recall, F1-score, and the Matthews Correlation Coefficient (MCC), chosen for its robustness in imbalanced scenarios. Results demonstrated highly efficient identification of DoS attacks (MCC 1.00), though Fuzzy DoS detection showed lower performance (MCC 0.214 in CAN-MIRGU and 0.074 in CAN-MODES). These findings support the viability of the proposed pipeline for IDS analysis focusing on enhancing CAN network security, consistent with recent research trends.