Modeling Allocation of Heterogeneous Storage Resources on HPC Systems - Archive ouverte HAL Accéder directement au contenu
Poster De Conférence Année : 2022

Modeling Allocation of Heterogeneous Storage Resources on HPC Systems

Modélisation d'allocation de ressources de stockage hétérogènes sur plateformes HPC

Julien Monniot
Gabriel Antoniu

Résumé

In recent years, and despite remarkable progress in computing and network performance, HPC platforms have struggled to maintain satisfactory I/O throughput. Various solutions have been proposed to mitigate the contention and variability experienced by more and more concurrent applications, particularly on heavily shared parallel file systems. As a result, many large scale platforms now offer complex hierarchies of storage resources using various architectures (node-local, burst buffers, network-attached storage and so on) based on diversified hardware technologies such as persistent memories or flash for instance. In that context, we propose to study how to efficiently allocate these heterogeneous storage resources for scientific applications and workflows. For that purpose, we developed StorAlloc, a modular and extensible simulator of a storage-aware job-scheduler. We ran a large set of experiments with StorAlloc to investigate storage system designs and resource scheduling algorithms, through the analysis of multiple storagerelated metrics. For example, we have been able to determine an estimate of the size of a burst buffers partition sufficient to capture the intensive I/O of a top-tier supercomputer.
Fichier principal
Vignette du fichier
rpost103s3.pdf (4.68 Mo) Télécharger le fichier
extended_abstract.pdf (79.49 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03878252 , version 1 (29-11-2022)

Licence

Identifiants

  • HAL Id : hal-03878252 , version 1

Citer

Julien Monniot, François Tessier, Gabriel Antoniu. Modeling Allocation of Heterogeneous Storage Resources on HPC Systems. SC 2022 - International Conference for High Performance Computing, Networking, Storage, and Analysis (Posters), Nov 2022, Dallas, United States. , pp.1-1. ⟨hal-03878252⟩
53 Consultations
75 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More