Domain-Guided Soft Actor–Critic for Network Slicing in Cell-Free Massive MIMO Systems

Abstract

Cell-free massive multiple-input multiple-output (mMIMO), which eliminates cell edge effects and enhances coverage and resource utilization, is suited for industrial Internet of things (IIoT) applications. In user-centric cell-free mMIMO-based IIoT networks, joint optimization of network slicing and access point (AP) selection is crucial for meeting diverse quality-of-service (QoS) requirements. However, the joint optimization is challenging due to the coupling of resource allocation decisions and typically imperfect channel state information. In this paper, we formulate the joint AP selection and network slicing problem as a constrained Markov decision process (CMDP) with a hybrid action space, and propose a deep reinforcement learning (RL) algorithm, domain-guided hybrid soft actor-critic for CMDP (DG-HSA2C), to maximize the long-term proportional fairness in UE transmission rates while ensuring their QoS across slices. DG-HSA2C integrates CMDP-based RL into a hybrid action space by extending the Lagrangian multiplier method. To mitigate reward hacking, our algorithm incrementally predicts future states and incorporates a domain-adaptation mechanism, enhancing fairness in resource allocation and balancing performance across slices. Simulations verify our algorithm’s effectiveness in achieving rate fairness among UEs and mitigating reward hacking under the balance of QoS and rewards.