DAPart: An Online DRL-based Adaptive Partition Framework for DNN Inference Acceleration and Energy Conservation in Edge Computing

Abstract

In an era dominated by data-driven solutions, Deep Neural Networks (DNNs), which have been proven to be pivotal tools in extensive applications across various domains, are evolving in terms of both depth and architecture to meet the escalating demands of contemporary utilizations. Nevertheless, deploying a complex DNN model on mobile devices may result in substantial processing latency and increasing energy consumption. The emerging Mobile Edge Computing (MEC), characterized by the allocation of computing capacity at the access point, enables the partitioning of DNN models so as to conserve energy on mobile devices and mitigate inference latency. Existing DNN partitioning methods typically train prediction models offline to make partition decisions and reduce end-to-end inference latency, which requires a great number of labeled datasets and may incur a prolonged pre-processing duration. In this paper, we develop an online Deep Reinforcement Learning (DRL) based adaptive partition method to dynamically determine optimal partitioning decision so as to jointly accelerate DNN inference and mitigate energy consumption. We run the proposed algorithm in an edge computing scenario consisting of NVIDIA Jetson Nano and an edge server equipped with RTX3090 for four different DNN models, including VGG16, MobileNetV2, ResNet50 and GPT2-medium. Then we collect actual processing latency and energy consumption and compare the performance of the proposed algorithm with state-of-the-art solutions. The experimental results demonstrate that, even under varying channel conditions, DAPart can achieve an average reduction of 38.8% in latency and 36.5% in energy consumption compared with other available methods.