Affinity-aware resource provisioning for long-running applications in shared clusters - Archive ouverte HAL
Article Dans Une Revue Journal of Parallel and Distributed Computing Année : 2023

Affinity-aware resource provisioning for long-running applications in shared clusters

Clément Mommessin
Renyu Yang
  • Fonction : Auteur correspondant
  • PersonId : 1303475

Connectez-vous pour contacter l'auteur
Satish Kumar
  • Fonction : Auteur
Jie Xu
  • Fonction : Auteur

Résumé

Resource provisioning plays a pivotal role in determining the right amount of infrastructure resource to run applications and target the global decarbonization goal. A significant portion of production clusters is now dedicated to long-running applications (LRAs), which are typically in the form of microservices and executed in the order of hours or even months. It is therefore practically important to plan ahead the placement of LRAs in a shared cluster so that the number of compute nodes required by them can be minimized to reduce carbon footprint and lower operational costs. Existing works on LRA scheduling are often application-agnostic, without particularly addressing the constraining requirements imposed by LRAs, such as co-location affinity constraints and time-varying resource requirements. In this paper, we present an affinity-aware resource provisioning approach for deploying large-scale LRAs in a shared cluster subject to multiple constraints, with the objective of minimizing the number of compute nodes in use. We investigate a broad range of solution algorithms which fall into three main categories: Application-Centric, Node-Centric, and Multi-Node approaches, and tune them for typical large-scale real-world scenarios. Experimental studies driven by the Alibaba Tianchi dataset show that our algorithms can achieve competitive scheduling effectiveness and running time, as compared with the heuristics used by the latest work including Medea and LraSched. Best results are obtained by the Application-Centric algorithms, if the algorithm's running time is of primary concern, and by Multi-Node algorithms, if the solution quality is of primary concern.
Fichier principal
Vignette du fichier
main.pdf (1015.64 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04271642 , version 1 (06-11-2023)

Identifiants

Citer

Clément Mommessin, Renyu Yang, Natalia Shakhlevich, Xiaoyang Sun, Satish Kumar, et al.. Affinity-aware resource provisioning for long-running applications in shared clusters. Journal of Parallel and Distributed Computing, 2023, 177, pp.1-16. ⟨10.1016/j.jpdc.2023.02.011⟩. ⟨hal-04271642⟩
33 Consultations
63 Téléchargements

Altmetric

Partager

More