Conference Papers Year : 2023

Condition-Based Maintenance Optimization Under Large Action Space with Deep Reinforcement Learning Method

Abstract

Effective maintenance is essential in keeping industrial systems running and avoiding failure. Condition-based maintenance (CBM) leverages the current degradation condition of the studied object to optimize future maintenance interventions. CBM optimization problems are complex for multi-component systems, facing the issue of the curse of dimensionality brought by the increase in the number of components. Reinforcement learning provides a promising perspective to overcome the issue. In this paper, we studied CBM optimization for a multi-component system in which the components degrade subject to the gamma process independently. We considered multiple maintenance choices for individual components, leading to a large combinatorial action space. In this case, traditional deep reinforcement learning algorithms like DQN may struggle to face the inefficiency of exploration. Instead, we propose exploiting Branching Dueling Q-network (BDQ), which incorporates the action branching architecture into DQN to drastically decrease the number of estimated actions. We trained a learning agent to minimize the expected cost for a long time horizon by taking maintenance actions according to the observed exact degrading signal for each component. We compared the policy learned by the agent with some other pre-defined static policies. The numerical results demonstrate the effectiveness of the learning algorithm and its potential for application in systems with more complex structures.
No file

Dates and versions

hal-04317702 , version 1 (01-12-2023)

Identifiers

Cite

Peng Bi, Yi-Ping Fang, Matthieu Roux, Anne Barros. Condition-Based Maintenance Optimization Under Large Action Space with Deep Reinforcement Learning Method. 6th International Conference on Optimization and Learning, OLA 2023, 2023, Malaga, France. pp.161-172, ⟨10.1007/978-3-031-34020-8_12⟩. ⟨hal-04317702⟩
82 View
0 Download

Altmetric

Share

More