Deep Reinforcement Learning-Based QoE Optimization for Heterogeneous Services in Satellite-Terrestrial Integrated MEC Networks

Abstract

The rapid proliferation of Internet of Things (IoT) and the diversity of services demand for a more efficient and intelligent resource allocation framework to enhance network performance. To this end, we construct a novel satellite-terrestrial integrated network (STIN) integrating multi-access edge computing (MEC) and millimeter wave (mmWave) technologies to explore the coordination gains of communication, caching, and computing resources from a perspective of joint optimization. To be specific, we first formulate the resource allocation issue of joint user association (UA), bandwidth allocation (BA), coded caching (CC), and computation allocation (CA), with the aim of maximizing the quality of experience (QoE) for heterogeneous services while guaranteeing diversified quality of service (QoS) requirements of user equipments (UEs). An alternating iterative optimization strategy is then developed, where convex optimization is applied to solve the CC and CA subproblems, while a multi-agent proximal policy optimization (MAPPO) algorithm is designed to jointly optimize UA and BA subproblem. Finally, extensive simulations demonstrate that our proposed algorithm achieves superior QoE performance compared to existing three benchmark algorithms.