Resource-Aware Scheduling of Distributed Machine Learning Jobs using Markov Decision Processes

Abstract

The efficient scheduling of distributed machine learning (ML) jobs presents a significant challenge due to the complex interplay of factors such as varying job resource requirements, heterogeneous computing environments, and dynamic workload fluctuations. Traditional scheduling approaches often rely on heuristics or simple optimization techniques, which may not effectively address the inherent complexities of ML workflows. This paper proposes a novel approach leveraging Markov Decision Processes (MDPs) to model and solve this scheduling problem. We formulate an MDP where states represent the current job queue and resource availability, and actions represent scheduling decisions, such as assigning a job to a specific worker or delaying its execution. A reinforcement learning (RL) algorithm is then employed to learn an optimal scheduling policy through interaction with the MDP. This approach offers a more principled and potentially more effective solution compared to traditional methods, leading to improved resource utilization, reduced job completion times, and overall enhanced performance of distributed ML systems. The core contribution lies in the formalization of the scheduling problem within an MDP framework and the subsequent application of RL to discover optimal scheduling strategies.