Design and implementation of an IoT-based multimodal stress monitoring architecture using edge AI and physiological sensors

Abstract

With the escalating increase in the number of stressful disorders, there is a need to design smart, intelligent, non-intrusive, and scalable solutions to support the monitoring of stress conditions in Internet of Thing (IoT) scenario. Here in this paper, an IoT-based multimodal stress sensing framework is presented with an embedded facial emotion recognition mechanism. The architecture of the proposed system can be recognized as a typical layer architecture for an IoT system: perception layer, edge intelligence layer, and application layer. The facial effect perception of the proposed IoT architecture is performed using a lightweight Convolutional Neural Network (CNN) classification approach to classify facial expressions into seven emotional states, while the physiological data are collected using a Photoplethysmography (PPG)-based pulse sensor connected to a microcontroller. The indicators are simultaneously processed to identify stress levels at the edge, thus providing better user privacy. The facial emotion recognition component was tested with the standard datasets like CK+, FER-2013, and FER+ with an accuracy of 99.47%, 84.35%, and 92%, respectively. Experimental validation of the facial emotion recognition component on benchmark data sets along with the real-time heart rate acquisition through the embedded PPG sensor proves the efficacy of the developed sensing modules to enable the use of the suggested IoT framework for stress monitoring purposes. The focus on “edge intelligence, affordable sensing, and modular Internet of things architecture” makes it inherently suitable for human-oriented Internet of things applications, such as workplace wellness monitoring and distant health tracking. The suggested light CNN architecture has a total of 4.48 million trainable parameters with the model size of around 17.14 MB, and thus computational efficiency and suitability for deployment in edge devices. In our research, validation was done on a laptop-edge computing system, whereas embedding this framework to constrained edge devices is left for future work. It can provide a good platform for stress tracking within an Internet of things environment.