Active RIS-Assisted Secure Beamforming for RSMA-ISAC System Based on Reinforcement Learning

Abstract

This paper investigates a robust and secure beamforming scheme for a reconfigurable intelligent surface (RIS) assisted integrated sensing and communication (ISAC) system, employing rate-splitting multiple access (RSMA). In the presence of a potential eavesdropper, the system aims to provide high-quality communication services to legitimate users while simultaneously performing radar sensing for target detection. To enhance physical layer security and spectral efficiency, we leverage the flexibility of RSMA to split messages into common and private streams, and utilize an RIS to reconfigure the wireless propagation environment. We formulate a joint optimization problem to maximize the radar output SNR of the system, subject to security rate constraints, total power budgets, and the nonconvex constraints of the RIS elements. Given the highly nonconvex nature of the problem and the dynamic environment, we propose a deep reinforcement learning (DRL) framework based on the deep deterministic policy gradient (DDPG) algorithm. Simulation results demonstrate that the proposed RSMA-ISAC scheme significantly outperforms traditional SDMA-based approaches in terms of secrecy rate and sensing accuracy, proving the effectiveness of the DRL-based joint beamforming design.