Demo: Multimodal Stress Monitoring with Split Federated Fine-Tuning on Mobile Devices
TL;DR
An end-to-end mobile system for proactive stress monitoring from wearable signals supported by an extensible split federated learning system that enables on-device LoRA fine-tuning of large language models.
Abstract
We present an end-to-end mobile system for proactive stress monitoring from wearable signals. The framework is supported by an extensible split federated learning system that enables on-device LoRA fine-tuning of large language models. By analyzing physiological data, the application distinguishes emotional states, summarizes recent trends, and provides brief, actionable suggestions. Upon detecting a negative emotional state, the paired smartwatch or smartphone proactively delivers notifications to users to encourage mindfulness. The on-site demonstration will showcase real-time sensing-inference on smartwatch-phone pairs, alongside a recorded split federated learning process that continuously learns from user private data. Our code has been open-sourced at: https://github.com/HKU-WILL-Lab/Multimodal-Stress-Monitoring-with-Split-Federated-Fine-Tuning-on-Mobile-Devices.
