Malleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities - Archive ouverte HAL
Article Dans Une Revue IEEE Transactions on Parallel and Distributed Systems Année : 2024

Malleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities

Ahmad Tarraf
Technische Universität Darmstadt - Technical University of Darmstadt
Martin Schreiber
Mathematics and computing applied to oceanic and atmospheric flows
Alberto Cascajo
Carlos III University of Madrid
Jean-Baptiste Besnard
ParaTools SAS
Marc-André Vef
Johannes Gutenberg - Universität Mainz = Johannes Gutenberg University
Dominik Huber
Technische Universität Munchen - Technical University Munich - Université Technique de Munich
Sonja Happ
ParTec AG
André Brinkmann
University Medical Center of the Johannes Gutenberg-University Mainz
David Singh
Carlos III University of Madrid
Hans-Christian Hoppe
Forschungszentrum Jülich GmbH | Centre de recherche de Jülich | Jülich Research Centre
Alberto Miranda
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Antonio Peña
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Rui Machado
Department High Performance Computing
Marta Garcia-Gasulla
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Martin Schulz
Technische Universität Munchen - Technical University Munich - Université Technique de Munich
Paul Carpenter
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Simon Pickartz
ParTec AG
Tiberiu Rotaru
Department High Performance Computing
Sergio Iserte
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Victor Lopez
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Jorge Ejarque
Barcelona Supercomputing Center - Centro Nacional de Supercomputacion
Heena Sirwani
Department High Performance Computing
Jesus Carretero
Carlos III University of Madrid
Felix Wolf
Technische Universität Darmstadt - Technical University of Darmstadt

Résumé

With the increase of complex scientific simulations driven by workflows and heterogeneous workload profiles, managing system resources effectively is essential for improving performance and system throughput, especially due to trends like heterogeneous HPC and deeply integrated systems with on-chip accelerators. For optimal resource utilization, dynamic resource allocation can improve productivity across all system and application levels, by adapting the applications’ configurations to the system’s resources. In this context, malleable jobs, which can change resources at runtime, can increase the system throughput and resource utilization while bringing various advantages for HPC users (e.g., shorter waiting time). Malleability has received much attention recently, even though it has been an active research area for almost two decades [1]. This paper presents the state-of-the-art of malleable implementations in HPC systems, targeting mainly malleability in compute and I/O resources. Based on our experiences, we state our current concerns and list future opportunities for research.

Domaines

Informatique

Martin Schreiber  : Connectez-vous pour contacter le contributeur

https://hal.science/hal-04689073

Soumis le : jeudi 5 septembre 2024-14:24:46

Dernière modification le : mercredi 18 décembre 2024-09:34:37

Consulter le texte intégral

Dates et versions

hal-04689073 , version 1 (05-09-2024)

Licence

Paternité

Identifiants

Citer

Ahmad Tarraf, Martin Schreiber, Alberto Cascajo, Jean-Baptiste Besnard, Marc-André Vef, et al.. Malleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities. IEEE Transactions on Parallel and Distributed Systems, 2024, 35 (9), pp.1551-1564. ⟨10.1109/TPDS.2024.3406764⟩. ⟨hal-04689073⟩

Exporter

BibTeX XML-TEI Dublin Core DC Terms EndNote DataCite

Collections

UGA CNRS INRIA INSMI LJK LJK_MAD LJK-MAD-AIRSEA INRIA2
54 Consultations
0 Téléchargements

Altmetric

Partager

More