Asynchronous Load Balancing and Auto-scaling: Mean-Field Limit and Optimal Design - Archive ouverte HAL Access content directly
Journal Articles IEEE/ACM Transactions on Networking Year : 2024

Asynchronous Load Balancing and Auto-scaling: Mean-Field Limit and Optimal Design


We develop a Markovian framework for load balancing that combines classical algorithms such as Power-of-d with auto-scaling mechanisms that allow the net service capacity to scale up or down in response to the current load on the same timescale as job dynamics. Our framework is inspired by serverless platforms, such as Knative, where servers are software functions that can be flexibly instantiated in milliseconds according to scaling rules defined by the users of the serverless platform. The main question is how to design such scaling rules to minimize user-perceived delay performance while ensuring low energy consumption. For the first time, we investigate this problem when the auto-scaling and load balancing processes operate asynchronously (or proactively), as in Knative. In contrast to the synchronous (or reactive) paradigm, asynchronism brings the advantage that jobs do not necessarily need to wait any time a scale-up decision is taken. In our main result, we find a general condition on the structure of scaling rules able to drive mean-field dynamics to delay and relative energy optimality, i.e., a situation where both the user-perceived delay and the relative energy waste induced by idle servers vanish in the limit where the network demand grows to infinity in proportion to the nominal service capacity. The identified condition suggests to scale up the current net capacity if and only if the mean demand exceeds the rate at which servers become idle and active. Finally, we propose a family of scaling rules that satisfy our optimality condition. Numerical simulations demonstrate that these rules provide better delay performance than existing synchronous auto-scaling schemes while inducing almost the same power consumption.
Fichier principal
Vignette du fichier
2204.02352.pdf (1.15 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04467913 , version 1 (20-02-2024)





Jonatha Anselmi. Asynchronous Load Balancing and Auto-scaling: Mean-Field Limit and Optimal Design. IEEE/ACM Transactions on Networking, 2024, pp.1-12. ⟨10.1109/TNET.2024.3368130⟩. ⟨hal-04467913⟩
13 View
3 Download



Gmail Facebook X LinkedIn More