A Triple-Verification Framework for Swift, Real-Time Detection of Anomalous Sensor Readings in Large-Scale IoT Systems

Abstract

Due to the massive expansion of Internet of Things systems and the variety of anomalous data resulting from faulty sensors, cyberattacks, or communication failures, it has become difficult to handle this type of data. Although current machine learning and statistical analysis techniques offer high accuracy in detecting anomalies, these methods alone are not sufficient for handling massive amounts of data in real-time. Therefore, the idea arose to propose a Triple-Verification Framework to detect and block anomalous data. The first stage of the framework acts as a firewall to block incoming messages from blocklisted/unauthorized sensors or carrying unknown schemes. In the second stage, a simple statistical analysis is performed to detect short-term anomalies such as out-of-range and stuck readings. In the third stage, a more advanced analysis is conducted to detect long-term anomalies such as calibration decay and gradual drift. To realistically evaluate the performance of the proposed framework under harsh and high-load conditions, 100,000 messages were sent to the framework via 2,000 simulated sensor nodes, each carrying a different type of 15 anomaly situations. Feedback played a pivotal role in achieving excellent results by implementing a proactive blocking mechanism to minimize computing resources and reduce latency to a minimum (in microseconds), in addition to adopting highly efficient technologies such as MQTT, Kafka, and MongoDB protocols to enable the framework to scale and exchange data efficiently.