AoI Minimization in Heterogeneous MEC Networks: A Federated Learning-Assisted Hybrid DRL and Convex Approach
This paper investigates a dynamic heterogeneous mobile edge computing network (HMECN), where mobile devices (MDs) could offload their full tasks to a small base station (SBS) directly or the macro base station (MBS) in direct or relay mode. As age of information (AoI) is a comprehensive and accurate metric to capture the freshness of computation results, we formulate a long-term weighted sum AoI (LWSA) minimization problem in the HMECN by jointly optimizing the offloading decisions of MDs as well as the bandwidth and computation resource allocation of all base stations, subject to energy, delay and peak AoI constraints. To address the formulated non-convex mixed integer nonlinear programming problem, we decompose it into the offloading decision optimization (ODO) top-problem and the resource allocation optimization (RAO) sub-problem. Based on the decomposition, we propose a federated learning (FL)-assisted hybrid DRL and convex approach that is comprised of a safe multi-agent DRL algorithm, convex optimization and FL. The ODO top-problem is solved by the safe multi-agent DRL algorithm, which strictly ensures that the actions of each agent do not exceed its energy constraint and then paves the way for using convex optimization to solve the RAO sub-problem. FL is used to alleviate the training instability problem aggravated by multi-agent settings via breaking the limitation of partial knowledge for each individual agent. Simulation results demonstrate the superiority of the proposed approach in terms of the LWSA, convergence, scalability and robustness in dynamic environments.