Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Concurrency and Computation: Practice and Experience Année : 2023

Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems

Résumé

Scaling up large-scale scientific applications on supercomputing facilities is largely dependent on the ability to scale up efficiently data storage and retrieval. However, there is an ever-widening gap between I/O and computing performance. To address this gap, an increasingly popular approach consists in introducing new intermediate storage tiers (node-local storage, burst-buffers, ...) between the compute nodes and the traditional global shared parallel file-system. Unfortunately, without advanced techniques to allocate and size these resources, they remain underutilized. In this paper, we investigate how heterogeneous storage resources can be allocated on an HPC platform, just like compute resources. To this purpose, we introduce StorAlloc, a simulator used as a testbed for assessing storage-aware job scheduling algorithms and evaluating various storage infrastructures. We illustrate its usefullness by showing through a large series of experiments how this tool can be used to size a burst-buffer partition on a top-tier supercomputer by using the job history of a production year.
Fichier principal
Vignette du fichier
SupportingDynamicAllocationOfHeterogeneousStorageResources.pdf (1.47 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY - Paternité

Dates et versions

hal-03922866 , version 1 (05-01-2023)
hal-03922866 , version 2 (13-11-2023)

Licence

Paternité

Identifiants

Citer

Julien Monniot, François Tessier, Matthieu Robert, Gabriel Antoniu. Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems. Concurrency and Computation: Practice and Experience, 2023, Special Issue:S2 World 2020. CUG 2021 & 2022. PN_HCP. HeteroPar 2022, 35 (28), pp.1-16. ⟨10.1002/cpe.7890⟩. ⟨hal-03922866v2⟩
114 Consultations
41 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More