A Two-Layer Multi-Agent Deep Reinforcement Learning Framework for Flexible Job-Shop Scheduling with Multiple Batch-Processing Machines

Abstract

Advancements in intelligent manufacturing require solutions to the conventional flexible job-shop scheduling problem (FJSP) to accommodate increasingly intricate constraints, particularly in the semiconductor and electronic component sectors, where batch-processing machines (BPMs) significantly intensify scheduling complexity. To address this challenge, this study formulates an extended FJSP with multiple BPMs and proposes an end-to-end two-layer multi-agent deep reinforcement learning framework. Job and machine agents perform decentralized action mapping, while workshop states are encoded using a heterogeneous disjunctive graph and a dual-graph attention network. Unlike standard FJSP learning methods that primarily address operation–machine decisions, the proposed framework jointly models machine assignment, operation sequencing, variable-length batch formation, and BPM allocation within a unified policy, with a pointer network-based batching agent and an equipment-selection agent that handle batch-processing decisions under feasibility masking. The framework was validated using plant-derived production data and multi-scale synthetic instances. Numerical results show that the proposed method achieves competitive performance across the tested batching and standard-FJSP settings. In standard-FJSP comparisons, relative performance was scenario-dependent: DANIEL performed better in S1, whereas both proposed variants outperformed all comparators in S2. These results support the framework as an effective scheduling approach for deterministic FJSP with BPMs and indicate cross-scale generalization across evaluated instances.