Hierarchical RSMA in Multi-Antenna Downlink Systems: Outage Performance and Power Allocation

Abstract

Rate-Splitting Multiple Access (RSMA) has emerged as a robust interference management strategy for future wireless networks. This paper investigates the performance of a hierarchical RSMA scheme in the downlink of a multi-antenna system, designed to efficiently serve clustered user deployments. We derive exact and asymptotic closed-form expressions for the outage probability of users under Nakagami- $m$ fading channels, considering a two-layer message splitting architecture (systemcommon, group-common, and private streams). Furthermore, to ensure fairness and reliability, we formulate a min-max power allocation problem to minimize the worst-case outage probability among users. A Geometric Programming-based algorithm is proposed to solve the resulting non-convex optimization problem. The numerical results validate the theoretical analysis and demonstrate the impact of different strategies for using this model, such as the number of users per group, user allocation strategies, and the number of base station transmit antennas.