Data-Driven Real-Time Scheduling Method Based on Deep Reinforcement Learning for IIoT-Enabled Discrete Manufacturing Workshop

Abstract

In a discrete manufacturing workshop with individualized production tasks and multi-disturbance production processes, real-time scheduling is urgently needed to shorten order completion time. Therefore, a real-time scheduling framework based on the industrial internet of things and deep reinforcement learning is proposed. Firstly, with the help of the industrial internet of things (IIoT), multi-source real-time manufacturing data are collected, and the information model and transmission protocol are unified by OPC Unified Architecture, which provides the IIoT environment for real-time scheduling. Secondly, 16 mixed scheduling rules are set to select work-in-process (WIP) in in-buffers for processing and to select a machine for handling the subsequent procedure of the WIP whose current operation is completed. A novel reward function is designed to guide the agent constructed by dueling double deep Q network to learn scheduling knowledge. Furthermore, in the exploration stage, noise search is used to select scheduling rules to improve the exploration ability, and mask processing is proposed to ensure the effectiveness of each exploration. In the utilization stage, the prioritized experience replay is adopted to select training samples to mine the scheduling knowledge fully. Finally, a machining workshop is taken as an example for verification. The results show that the proposed method can optimally and universally solve the scheduling problem of the discrete manufacturing workshop compared with the current optimization methods.