Multi-Metric Explainability Assessment of Machine Learning Models for Physical Access Control

Abstract

This paper presents a structured and quantitative framework for evaluating the explainability of machine learning (ML) models trained on synthetic access control datasets. Explainability is assessed using four complementary metrics - stability, consistency, fidelity, and monotonicity - along with a normalized composite explainability score $E_{\text{norm}}$ that enables balanced comparison across models and datasets by addressing scale incompatibility among metrics. The framework is evaluated across four ML models (Logistic Regression, k-Nearest Neighbors, Naive Bayes, and a Neural Network) trained on six synthetic datasets ranging from rule-based and probabilistic to random and adversarial generation strategies. Results show that simpler models consistently produce more stable, consistent, and monotonic explanations on structured synthetic data, while neural networks exhibit greater variability and over-reliance on dominant features. Structured data generation methods were found to support stronger explainability than unstructured or adversarial approaches. Together, these findings provide practical guidance for selecting model-dataset combinations that support transparent and auditable decision-making in physical access control systems.