Generative AI Service Provision in Heterogeneous Edge Networks: A Dynamic Two-Stage Optimization Approach

Abstract

Generative Artificial Intelligence (GAI) has been attracting a massive and rapidly expanding user base worldwide in recent years, resulting in enormous demand for inference requests that cannot be handled efficiently by centralized cloud-based architectures. Edge computing presents a promising approach to mitigate these challenges by leveraging the power of numerous edge devices to better provide GAI services to the users. In this work, we develop a novel two-stage approach to dynamically allocate GAI models and assign user requests to the best edge nodes. In the first stage, we model a joint optimization problem to minimize the expected processing time and decide the optimal model allocation based on predicted user demands. In the second stage, we develop an efficient online approach to assign requests to edge nodes when they arrive, as well as to reallocate GAI models when necessary. Moreover, to address the complexity of the optimization problems in this stage, we leverage Lyapunov optimization framework and Benders decomposition methods to efficiently solve the problems, thereby enabling the proposed approach to quickly adapt to the dynamics of the system. Extensive simulations are conducted to evaluate the performance of the proposed approach and investigate the impacts of important parameters. Simulation results show that the proposed approach can reduce the total processing time by up to 43% with very short running time.