Virtualization of accelerators in cloud-edge computing

Abstract

Edge computing has the advantage of harnessing compute capabilities on remote resources located at the edge of the network to run workloads of relatively weak user devices. On the other hand, cloud computing offers compute capabilities on remote resources located, generally, geographically far from user devices on substantially more powerful devices, compared to the edge. Combining two unique paradigms allows for mutual strengthening. I.e., where one paradigm may be weak, the other is strong and vice versa. On the one hand, workloads such as Deep Learning may require large amounts of computational power to be executed. On the other hand, user devices such as Internet of things (IoT) devices or low-powered embedded devices may not contain the computational capabilities required to complete workloads such as Deep Learning. However, by offloading tasks to the edge-cloud continuum, these computationally weaker user devices can access the computational capabilities of more powerful remote nodes. One such method for achieving this is remote accelerator virtualization. This technique allows user devices to make use of the accelerator stored on remote nodes to complete their tasks. This thesis demonstrates the potential for accelerator virtualization, specifically Deep Learning inference-based workloads, in the edge-cloud continuum, which consists of user devices, edge nodes and the cloud itself. Within this thesis, the focus is on graphics processing unit (GPU) accelerators. Although there are machine learning-specific accelerators available, the sheer popularity, availability and support for GPUs make them an enticing choice. Achieving accelerator virtualization in edge-cloud computing requires challenges to be overcome. Edge-cloud computing is heterogeneous in nature with multiple types of hardware and software capabilities potentially being present within the network. Applications may have latency restrictions or the dynamic nature of the edge-cloud continuum may result in remote nodes being unavailable. These problems and others are addressed in this thesis.