Penalty-Free GNN Framework for Joint Optimization in RIS-Assisted ISAC Systems

Abstract

Jointly optimizing base station beamforming and reconfigurable intelligent surface (RIS) phase shifts in integrated sensing and communication (ISAC) systems under strict sensing constraints presents a challenging non-convex problem. Conventional algorithms suffer from high complexity, while penalty-based deep learning struggles to guarantee strict constraint satisfaction. To address this, we propose a novel penalty-free unsupervised Graph Neural Network (GNN) framework. Crucially, we introduce a beam decomposition and reconstruction layer that projects beamforming vectors into the sensing subspace and its orthogonal complement. By separating and scaling the sensing-aligned and orthogonal components, this mechanism theoretically guarantees 100% satisfaction of sensing signal-to-noise ratio and total power constraints without hyperparameter tuning. Simulation results demonstrate that the proposed approach ensures strict sensing performance guarantee and outperforms both traditional iterative algorithms and penalty-based baselines in sum-rate, while maintaining ultra-low inference time across different RIS sizes, thereby enabling real-time scalability.