Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling

Résumé

High-performance computing is not only a race towards the fastest supercomputers but also the science of using such massive machines productively to acquire valuable results-outlining the importance of performance modelling and optimization. However, it appears that more than punctual optimization is required for current architectures, with users having to choose between multiple intertwined parallelism possibilities, dedicated accelerators, and I/O solutions. Witnessing this challenging context, our paper establishes an automatic feedback loop between how applications run and how they are launched, with a specific focus on I/O. One goal is to optimize how applications are launched through moldability (launch-time malleability). As a first step in this direction, we propose a new, always-on measurement infrastructure based on state-of-the-art cloud technologies adapted for HPC. In this paper, we present the measurement infrastructure and associated design choices. Moreover, we leverage an existing performance modelling tool to generate I/O performance models. We outline sample modelling capabilities, as derived from our measurement chain showing the critical importance of the measurement in future HPC systems, especially concerning resource configurations. Thanks to this precise performance model infrastructure, we can improve moldability and malleability on HPC systems.
Fichier principal
Vignette du fichier
hpcmall.pdf (584.98 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04093528 , version 1 (10-05-2023)

Licence

Identifiants

  • HAL Id : hal-04093528 , version 1

Citer

Jean-Baptiste Besnard, Ahmad Tarraf, Clément Barthélemy, Alberto Cascajo, Emmanuel Jeannot, et al.. Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling. HPCMALL 2023 - 2nd International Workshop on Malleability Techniques Applications in High-Performance Computing, May 2023, Hamburg, Germany. ⟨hal-04093528⟩
129 Consultations
41 Téléchargements

Partager

More