On DNN Model Partitioning and Deployment for Collaborative Multi-Edge AI Computing

Abstract

Offloading DNN workload to an edge network for parallel and distributed computing is a promising solution to hosting computation-intensive DNN services. The challenge is how to partition DNN models and to reduce the communication to realize a high-performance multi-edge DNN computing. In this paper, we first propose a novel two-stage bayesian optimization based approach to DNN model partitioning, which can decouple a DNN model into components at appropriate granularity while at the same time incur less communication cost between them. The proposed approach is physical infrastructure agnostic and can achieve a decomposition ratio of 3.2× to 35.0× on popular DNN models while respecting their execution parallelism. Moreover, we consider online DNN model deployment scenarios, with the aim to maximize the rate of requests supported by the network. We formulate a mixed integer linear programming (MILP) problem for the task and establish its NP-hardness. To tackle it efficiently, we decompose the problem into two sub-problems, and propose a binary search method to the upper-level one and a backtracking based algorithm to the lower-level one. We evaluate our mechanisms through both simulation and experiments, and results show they are effective as compared to baselines.