A Q-learning approach to planning a sustainable food manufacturing system
Résumé
For the agri-food sector, the third most energy-intensive industrial sector, one of the major challenges is to reduce its carbon footprint and ensure sustainable manufacturing to limit its impact on climate change. In addition, recent global market instabilities with soaring energy prices have reinforced the need for the food sector to conduct an effective energy and transition policy. To meet this challenge, one of the alternatives is to replace carbon intensive processes with low-carbon processes involving electrical energy together with renewable energy as main sourcing energy for food processing. Such an evolution implies: (1) to consider the amount of the different sources of energy needed for a production plant and (2) to optimize energy costs by diversifying sources. Within this scope, this study aims at developing a novel scheduling approach integrating renewable energy use to meet demand over a limited planning horizon with the lowest possible costs for inventories, backorders, production setup and energy consumption. In particular, a food industrial scheduling problem is investigated with sequence-dependent that extends multi-products and multi-process capacitated lot sizing in a flexible flow line, in which non-identical machines works in parallel. An advanced mixed integer linear program adapted to renewable energy supply is proposed to provide an hourly production schedule, which minimizes energy and production costs. To bypass consuming execution time limitations and to realize the industry 4.0 vision for production optimization, an algorithm based on reinforcement learning is considered. This machine learning approach has gained in popularity in dynamic optimization problems since the last decade. Hence, it turns out to associate each machine of the process to an agent trained to minimize total cost function and to adopt cooperative multi-agent strategy to solve the global problem. Numerical experiments are performed on a practical reference case. First, the efficiency of the proposed algorithm is evaluated against other advanced techniques such as metaheuristics. Then, the interest of the approach is discussed in terms of energy and financial improvements