A reinforcement learning framework for energy-aware semiconductor manufacturing operations

Abstract

Semiconductor manufacturing is among the most energy-intensive industrial activities because its production processes are highly complex, equipment is used at high levels, and operations must run continuously. In many traditional manufacturing settings, control strategies mainly aim at productivity—such as throughput, cycle time, and equipment utilization—while paying less attention to energy efficiency and sustainability. However, as electricity prices keep increasing and carbon-reduction requirements become more urgent, it has become a major challenge to design intelligent operating strategies that meet production goals while also addressing energy-related objectives. In this study, we present a reinforcement learning–based framework for energy-aware semiconductor manufacturing operations. The manufacturing process is formulated as a sequential decision-making problem, where the production state—such as work-in-process levels, queue lengths, equipment utilization, production demand, electricity prices, and carbon intensity—is observed continuously. Using the current system state, the reinforcement learning agent makes decisions in real time, including lot release control, equipment mode selection, and production priority assignment. We also construct a reward function that accounts together for production throughput, energy cost, carbon emissions, queue congestion, and delivery delays. This allows the learning agent to develop operational policies that strike a balance between manufacturing performance and sustainability targets. To verify the proposed framework, we build a simulation-based semiconductor manufacturing environment and compare it with several benchmark operational policies. The experimental results show that the reinforcement learning approach can significantly lower overall operational cost and energy consumption while still delivering acceptable throughput performance. Overall, the proposed framework offers a practical decision-support tool for sustainable semiconductor manufacturing and provides useful guidance for intelligent energy management in future smart factory systems.