A SETUP-AWARE DEEP REINFORCEMENT LEARNING FRAMEWORK FOR SINGLE-MACHINE SCHEDULING WITH SEQUENCE-DEPENDENT SETUP TIMES USING PROXIMAL POLICY OPTIMIZATION (PPO)

Abstract

The Single-Machine Scheduling Problem with Sequence-Dependent Setup Times (SMSP-SDST) is a classical NP-hard combinatorial optimization problem encountered in intelligent manufacturing, production planning, semiconductor processing, logistics, and adaptive production systems. Conventional exact optimization techniques become computationally prohibitive for large scheduling instances, whereas heuristic and metaheuristic approaches often exhibit limited adaptability under dynamically changing production environments. This study proposes a setup-aware DRL framework for solving SMSP-SDST through adaptive sequential decision-making. The scheduling problem is described as a Markov decision process (MDP) in which the states encode unscheduled jobs, machine progression, processing characteristics, and sequence-dependent setup transitions. A PPO agent incorporating action masking and reward shaping is developed to learn feasible and setup-efficient scheduling policies. The proposed framework is evaluated using Taillard benchmark-inspired scheduling instances and compared against the following algorithms: Genetic Algorithm, Simulated Annealing, Tabu Search, NEH heuristic, Greedy-MinSetup heuristic, and Random scheduling policies. The DRL scheduler achieves superior scheduling performance for small- and medium-scale instances, reducing the average makespan for 20-job instances by approximately 2.7% relative to NEH and 0.9% relative to GA while maintaining improved setup efficiency. This study contributes a reproducible AI-driven scheduling methodology for intelligent production scheduling under sequence-dependent setup constraints that integrates setup-aware representation learning, MDP formulation, and PPO-based policy optimization