Factored Reinforcement Learning for Auto-scaling in Tandem Queues - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Factored Reinforcement Learning for Auto-scaling in Tandem Queues

Apprentissage par renforcement factorisé pour l'autoscaling dans un modèle de file en tandem


As today’s networking systems utilize more virtualisation, efficient auto-scaling of resources becomes increasingly critical for controlling both the performance and energy consumption. In this paper, we study the techniques to learn the optimal auto-scaling policies in a distributed network when parts of the system dynamics are unknown. Reinforcement Learning methods have been applied to solve auto-scaling problems. However they can run into computational and convergence issues as the problem scale grows. On the other hand, distributed networks have relational structures with local dependencies between physical and virtual resources. We can exploit these structures to overcome the convergence issues by using a factored representation of the system.We consider a distributed network in the form of a tandem queue composed of two nodes. The objective of the auto-scaling problem is to find policies that have a good trade-off between quality of service (QoS) and operating costs. We develop a factored Reinforcement Learning algorithm, named FMDP online, to find the optimal auto-scaling policies. We evaluate our algorithm with a simulated environment. We compare it with existing Reinforcement Learning methods and show its relevance in terms of policy efficiency and convergence speed.
Fichier principal
Vignette du fichier
Factored_Model_Based_RL_TQ.pdf (624.97 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03811176 , version 1 (16-02-2023)



Thomas Tournaire, Yue Jin, Armen Aghasaryan, Hind Castel-Taleb, Emmanuel Hyon. Factored Reinforcement Learning for Auto-scaling in Tandem Queues. NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium, Apr 2022, Budapest, Hungary. pp.1-7, ⟨10.1109/NOMS54207.2022.9789809⟩. ⟨hal-03811176⟩
37 View
22 Download



Gmail Facebook Twitter LinkedIn More