Optimising Aircraft Fleet Maintenance with Reinforcement Learning: A numerical case study

Abstract

Unexpected failures and inefficient maintenance plans lead to costly downtime in many industrial fields, undermining operational sustainability and overall profitability, or, in the worst scenarios, creating unsafe situations for people. These issues justify a shift towards predictive maintenance, and structural health monitoring (SHM) has emerged as a viable option. By exploiting modern hardware and software technologies, the diagnosis and prognosis of monitored structures can be performed while accounting for estimation uncertainty. In addition, for complex structures, monitoring all components is not feasible for technical or economic reasons, and a mix of condition and schedule-based maintenance approaches is still required. To fully exploit the SHM benefits, an effective decision-making process is necessary, including Opportunistic Maintenance (OM) that accounts for both scheduled stops and the Residual Useful Life (RUL) predicted by prognostic models. In this context, decision-making becomes significantly more complex when managing an entire fleet rather than a single asset, reflecting the real-world challenges companies face. This study explores the application of Reinforcement Learning (RL) to automated decision-making in aircraft fleet maintenance, aiming to minimise operational and maintenance (O&M) life-cycle costs. The RL agent is trained and deployed within a fleet life cycle simulator that replicates real-world conditions, including mission schedules, maintenance operations, repair bases with varying capabilities, and potential unforeseen events. Each aircraft is modelled as an ensemble of subcomponents, with only a portion equipped with an SHM system that provides Remaining Useful Life (RUL) predictions, expressed as probability distributions, and fault detection. The agent leverages this data to develop an optimal maintenance policy, aligning scheduled stops for non-monitored components with necessary interventions for monitored parts. Specifically, it determines maintenance schedules and selects which components to replace based on fleet-wide data, expected mission schedules, and aircraft availability. While the study is demonstrated using a fleet life-cycle simulator, it serves as a preliminary step toward the real-world implementation of autonomous agents. The integration of the agent within the fleet simulator, potentially based on a real scenario, enhances its capabilities as a Digital Twin, supporting management decisions. Furthermore, it leverages the potential benefits of SHM technology, encouraging broader industry adoption.