Reinforcement learning-assisted time-to-boundary mobility control with selective DAPS for QoS-aware handover in dense HetNets

Abstract

This paper presents a hybrid rule-constrained and reinforcement learning-assisted Time-to-Boundary (TTB)-guided mobility-control framework with selective Dual Active Protocol Stack (DAPS) activation for energy-efficient QoS-aware handover in dense heterogeneous networks. The proposed one-dimensional (1-D) boundary-crossing model uses predicted boundary arrival time, radio-link condition, handover margin, and queue-risk indicators to suppress unnecessary measurement signaling and activate DAPS or handover only when mobility or QoS risk is detected. The controller combines rule-based feasibility masks, safety overrides, handover confirmation, and post-handover lockout with a lightweight tabular Q-learning policy. The rule-based layer restricts unsafe or infeasible actions, while the RL layer selects among feasible idle, measurement, selective-DAPS, and handover actions according to the current radio, boundary, and queue state. The proposed scheme is evaluated against conventional A3-Time-to-Trigger (A3-TTT), optimized A3-TTT, persistent-DAPS A3-TTT, and conditional handover schemes using paired multi-seed simulations with confidence intervals and statistical testing. Results show that OURS-RL-QTAB provides a favorable QoS–energy–signaling trade-off under the controlled simulation setting by reducing mean delay, 95th-percentile delay, packet loss, signaling load, and normalized energy cost per delivered Mbit while maintaining comparable throughput and utilization. Persistent-DAPS achieves the lowest jitter, but at substantially higher energy cost and much higher signaling load than the proposed talk-less controller. Sensitivity, generalization, and ablation analyses further confirm that the proposed performance is not dependent on a single parameter setting and that TTB prediction, talk-less operation, selective DAPS, queue-risk awareness, safety triggering, reinforcement learning action selection, and handover lockout each contribute to the final performance. Within the controlled single-UE 1-D boundary-crossing model, the results indicate a favorable overall QoS–energy–signaling trade-off for selective risk-aware DAPS activation with hybrid rule-constrained and RL-assisted control. Not applicable. This study does not involve clinical trials, human participants, or medical interventions and is based entirely on simulation and computational modeling.