Aug 2026· Processes· Vol 14, pp. 2519· 0 citations· 31 references
TL;DR
This paper proposes ALP-PPO, an adaptive Lagrangian penalty-enhanced proximal policy optimization algorithm for real-time rescheduling under concurrent machine breakdowns and rush orders, and indicates that the adaptive Lagrangian mechanism reduces constraint violations by more than 40% relative to fixed-penalty alternatives while keeping the primary objectives competitive.
Abstract
When flexible job shop scheduling faces concurrent disturbances such as machine failures and rush orders, worker-centric constraints emphasized under Industry 5.0 must also be satisfied. Existing deep reinforcement learning methods for the Dynamic Flexible Job Shop Scheduling Problem (DFJSP) seldom treat worker workload balance as an explicit constraint, and most depend on static penalty coefficients that are difficult to tune across different scenarios. In this paper, we suggest ALP-PPO, an adaptive Lagrangian penalty-enhanced proximal policy optimization algorithm, for real-time rescheduling under concurrent machine breakdowns and rush orders. We formulate the scheduling environment as a constrained Markov decision process. Worker skill heterogeneity, fatigue accumulation and workload equity are modeled as coupled constraints alongside classical scheduling objectives. By decoupling operation sequencing, machine allocation and worker assignment into coordinated sub-decisions, a hierarchical action space is constructed. Dual Lagrangian multipliers for workload balance and fatigue are updated adaptively during training, so that manual penalty tuning is no longer required. An event-triggered mechanism selects between right-shift and full rescheduling on the basis of a disruption severity index. We employ weighted-sum scalarization of makespan, energy consumption and workload variance during training, and Pareto solution sets are obtained by systematically varying the weight vectors across independent training runs. On extended Brandimarte benchmarks augmented with worker and dynamic event parameters, ALP-PPO delivers superior scheduling performance across makespan, energy consumption and workload variance when compared with Double DQN, Dueling DQN, standard PPO, NSGA-II and MOEA/D, as measured by Hypervolume (HV) and Inverted Generational Distance (IGD) indicators. Ablation studies indicate that the adaptive Lagrangian mechanism reduces constraint violations by more than 40% relative to fixed-penalty alternatives while keeping the primary objectives competitive. An analysis of computational efficiency shows that ALP-PPO completes online inference in under 20 ms per decision step, making real-time rescheduling practically feasible. Generalization experiments on previously unseen instances further validate the transferability of the learned policy. These findings support human-centric intelligent scheduling in Industry 5.0 manufacturing.
This work addresses the resulting Multi-Objective Flexible Job Shop Scheduling Problem by proposing a deep reinforcement learning framework that jointly minimizes makespan and energy cost, and evaluates the approach against NSGA-II and Joined Heuristics on synthetic instances.
Dustin Moreira Simoes, Marvin Brune, Mehmet Ulrich et al.· Applied Sciences· 0 citations
PGMPO is proposed, a novel learning framework consisting of a simple but effective multi-policy modeling approach that allows a single network to represent multiple decision-makers, and a preference-driven model optimization method that effectively guides policies to learn diverse and specialized problem-solving strategies without the need for explicit reward functions.
Inguk Choi, Woo-Jin Shin, Sang-Hyun Cho et al.· 0 citations
Job scheduling in Electronic Design Automation (EDA) environments presents unique challenges due to high-frequency job submissions, short job durations, and strict latency requirements. Production schedulers such as IBM Spectrum LSF employ robust heuristics like First-Come-First-Served (FCFS) that provide predictable behavior and fairness guarantees. However, these rule-based approaches do not learn from historical workload patterns, leaving potential for further optimization through adaptive methods. We present Adaptive Job Selection (AJS), a reinforcement learning-based agent that learns to schedule pending jobs for reduced job waiting and completion time on LSF clusters for EDA workloads. AJS introduces macro actions that dispatch multiple jobs per inference, addressing the credit assignment problem inherent in high-frequency scheduling while meeting real-time latency constraints. Our lightweight neural network architecture employs cross-attention to capture interactions between job buckets and cluster state, enabling inference at the frequency required by EDA workloads. We deploy AJS as an external plugin in IBM Spectrum LSF and evaluate it on an IBM LSF cluster. Experiments demonstrate that AJS achieves a 62.6% reduction in average job waiting time, a 19.8% reduction in job completion time, a 16.0% improvement in job throughput, and 3.05 percentage points higher CPU utilization compared to the default scheduler. We also share practical lessons for bridging the simulation-to-production gap. To our knowledge, AJS is the first open-source, deployable RL-based scheduler designed for production EDA environments.
Yiming Shao, Aijun An, Michael Spriggs et al.· Proceedings of the 32nd ACM...· 0 citations
Most learning-based schedulers for job shop problems assume static workforce performance. However, dynamic dual-resource job shops must navigate stochastic disturbances and time-varying task durations due to human factors. This study proposes a human-centered Deep Reinforcement Learning framework, DD4LQN, for dynamic flexible job shop scheduling under operator learning and forgetting dynamics. The environment integrates a disturbance-aware scenario generator, bounded logistic learning-forgetting dynamics, and deterministic Dual Pair Ranking to ensure auditable decisions. The training used a plan that included ENTRY and EXIT greedy evaluations and a Quote-then-Commit process. Tests over ten weeks showed that DD4LQN-EXIT did better than other scheduling methods. Empirical evaluations over a ten-week horizon demonstrate that DD4LQN-EXIT got the average schedule reward of 0.5843. It was better than Earliest Due Date (EDD) and Shortest Processing Time (SPT) by 17% and 14%, respectively. It also completed jobs and had fewer risky orders. Even though Shortest Processing Time had delays on average, it completed fewer jobs because it focused on short tasks. Additionally, an exploratory ablation across the same scenarios demonstrated that Dual Pair Rank and learning–forgetting dynamics come up with better results. Furthermore, representation diagnostics on the 227,717-parameter network confirm structural stability, with layer matrices retaining up to 98.2% of maximum effective rank without capacity collapse.
Unknown authors· International Journal of Adv...· 0 citations
Analysis of queue-level dynamics reveals more regular behavior in the evaluated scenarios, with reduced fluctuations in queue lengths, batch waiting, and minimum queue entropy over time, indicating that the proposed ABC-based approach can improve observed predictability at the queue level.
Human–robot collaborative flexible job shop scheduling (HRC-FJSP) must coordinate heterogeneous capabilities, mode-dependent processing times, safety feasibility, and carbon constraints. The problem becomes harder when a collaboration mode that is attractive during planning becomes infeasible after a human enters the robot safety separation zone. Unlike conventional dynamic disturbances such as machine breakdown or order insertion, this event changes the feasible collaboration mode of the unfinished operation remainder rather than only delaying a resource or adding a job. This study formulates a carbon-aware dynamic HRC-FJSP and evaluates a carbon-aware multi-agent deep reinforcement learning scheduler (CA-MADRL) with local recovery after safety-proximity-induced collaboration disruption. The objective combines normalized makespan, carbon emission, and human workload imbalance with carbon accounting based on operation energy and time-varying grid carbon intensity. Across the benchmark cases, CA-MADRL obtains the best average global criterion (0.7235), wins nine of 12 cases, and achieves the lowest average carbon emissions among the compared policies (48.991 kg CO2e). Sensitivity analysis shows that stronger carbon preference reduces emissions but increases makespan and tardiness, while adaptive collaboration outperforms fixed human–robot, human-only, and robot-only regimes. The results indicate that dynamic mode adaptation and local rescheduling improve carbon-aware collaborative schedules under safety disruption.
