Trans-SAC: gamification design for sustained user engagement in residential demand response

Abstract

Abstract Residential Demand Response (DR) is a critical mechanism for maintaining the supply-demand balance in modern smart grids. While gamification has recently emerged as a promising strategy to incentivize residential participation, existing programs suffer from severe long-term user fatigue. Once the initial "novelty effect" wears off, users rapidly habituate to static rewards, leading to a precipitous decline in DR reliability. Furthermore, traditional pricing models and standard Deep Reinforcement Learning (DRL) agents optimize purely for physical grid states, failing to capture the latent psychological dynamics of human participants.To address this critical gap, this paper proposes Trans-SAC, a novel gamification-aware dynamic DR framework that integrates psychological fatigue modeling with advanced machine learning. First, we formulate a mathematical behavioral model to quantify the accumulation and decay of user fatigue. Second, a Transformer-based module is employed to process long-sequence historical interaction logs, utilizing multi-head self-attention to predict the real-time cognitive engagement probability of each user. Finally, this psychological state is incorporated into the state space of a Soft Actor-Critic (SAC) agent. Guided by entropy-regularized exploration, the SAC agent autonomously executes "intermittent reinforcement" by dynamically adjusting task difficulties and reward multipliers.Extensive simulations, which combine real-world electricity-consumption traces from the Pecan Street dataset with a synthetically generated gamification-behavior environment, demonstrate the potential effectiveness of the proposed framework. Over a continuous 12-month simulated period, Trans-SAC was found to mitigate the modeled user-habituation process, maintaining a simulated Load Reduction Achievement Rate (LRAR) of 85.6% and a simulated User Retention Rate of 84.2%. Crucially, by strategically restraining rewards for intrinsically motivated users and deploying high-value interventions only when critical fatigue is detected, Trans-SAC achieved a Unit Incentive Cost of $0.061/kWh, representing a 21.8% economic improvement over standard DRL baselines under the simulated behavioral assumptions detailed in Section 4.1.The results suggest that coupling cognitive state assessment with DRL achieves a favorable trade-off between long-term grid reliability and economic efficiency under the simulated behavioral assumptions of this study.