Charging Optimization for Mobile Devices With Multi-Agent Reinforcement Learning in Wireless Rechargeable Sensor Networks

Abstract

Wireless power transfer (WPT) technique is promising for addressing the energy bottleneck of conventional wireless sensor networks (WSNs). Most existing work focuses on optimizing either the deployment of static chargers or the trajectory design of mobile chargers when the device-to-be-charged is static. However, when devices are mobile, the charging optimization becomes more challenging and remains insufficiently explored. In this work, we investigate the charging optimization for mobile devices, where some devices move along their scheduled routes and some static wireless chargers are deployed to charge the devices within a certain area. The objective is to minimize the time-averaged total power consumption of all chargers over a finite time horizon, while minimizing the time-averaged proportion of devices whose real-time battery levels fall below a given safety threshold. To address the spatiotemporal charging, partial observability of device-side information, and trade-off between chargers’ power minimization and devices’ battery level maintenance, we propose D2C-MASAC, a decoupled-dual constrained multi-agent soft actor-critic algorithm that combines decoupled policy learning with adaptive Lagrangian adjustment under constrained multi-agent reinforcement learning (MARL). Simulation-based validation is conducted in a 100m $\times 100$ m wireless rechargeable sensor network (WRSN) with 9 or 12 static chargers and 5, 10, or 20 mobile devices. The training process uses three random seeds and the final evaluation uses 50 independent test episodes. We define non-equal-weighted (NEW) and equal-weighted (EW) composite scores over three evaluation metrics, i.e., the average total power, the average safety threshold violation ratio, and the distribution of device battery levels. D2C-MASAC outperforms six MARL baselines and six optimization-based or heuristic-based benchmarks, achieving NEW/EW gains of 26.66%/28.07% and 35.96%/40.12%, respectively.