Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems

Résumé

Scaling up large-scale scientific applications on supercomputing facilities is largely dependent on the ability to scale up efficiently data storage and retrieval. However, there is an ever-widening gap between I/O and computing performance. To address this gap, an increasingly popular approach consists in introducing new intermediate storage tiers (node-local storage, burst-buffers, ...) between the compute nodes and the traditional global shared parallel file-system. Unfortunately, without advanced techniques to allocate and size these resources, they remain underutilized. In this paper, we investigate how heterogeneous storage resources can be allocated on an HPC platform, just like compute resources. To this purpose, we introduce StorAlloc, a simulator used as a testbed for assessing storage-aware job scheduling algorithms and evaluating various storage infrastructures. We illustrate its usefullness by showing through a large series of experiments how this tool can be used to size a burst-buffer partition on a top-tier supercomputer by using the job history of a production year.
Fichier principal
Vignette du fichier
StorAlloc_Extended.pdf (1.34 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03922866 , version 1 (05-01-2023)
hal-03922866 , version 2 (13-11-2023)

Identifiants

  • HAL Id : hal-03922866 , version 1

Citer

Julien Monniot, François Tessier, Matthieu Robert, Gabriel Antoniu. Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems. 2022. ⟨hal-03922866v1⟩
118 Consultations
44 Téléchargements

Partager

Gmail Facebook X LinkedIn More