rCamInspector: Building Reliability and Trust on IoT (Spy) Camera Detection using XAI

Abstract

The classification of network traffic using machine learning (ML) models has become popular to address the security issues in IoT networks and/or IoT devices. However, the classifiers often act as black-boxes which hinders its adoption in critical decision making. To address this problem, we design and develop a system, called rCamInspector, that employs Explainable AI (XAI) to generate reliable and trustworthy explanations for the model outputs. It adopts two classifiers, Flow Classifier and SmartCam Classifier; both of them are IP address and transport port agnostic. rCamInspector is evaluated using a total of 38 GB network traffic collected over 72 hours. Our results show that Extreme Gradient Boosting (XGB) achieves the highest accuracy of 92% and 99% in the Flow and Smart-Cam classifiers respectively, among eight supervised ML models. We empirically show that the traditional mutual information (MI) based important features in XGB remain almost same in either classifier. Using SHAP and LIME, we show that the most important features in Flow Classifier are significantly different from those in the SmartCam Classifier for the correct prediction using XGB. To evaluate the faithfulness of SHAP and LIME on the dataset, we show that both have a consistency of more than 0.7 and a sufficiency of 1.0. Compared to the existing works, rCamInspector achieves a better accuracy (99%), precision (99%), and false negative rate (0.7%), supported by the XAI for the predictions.