Modeling Multi-Threaded Aggregated I/O for Asynchronous Checkpointing on HPC Systems - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Modeling Multi-Threaded Aggregated I/O for Asynchronous Checkpointing on HPC Systems

Bogdan Nicolae
Jon Calhoun
  • Fonction : Auteur

Résumé

HPC systems encompass more components with each new generation. As a result, the process of interacting with stable storage systems like parallel file systems (PFS) becomes increasingly difficult. Larger systems often result in more frequent failures, increasing the need and frequency to incorporate fault-tolerant mechanisms. One example is checkpoint-restart (C/R), where applications or systems save their data to nonvolatile storage devices, such as a PFS. On failure, the system or application is restored to a saved state and computation continues. Today, asynchronous C/R is gaining traction for its ability to checkpoint data to permanent storage concurrently with the application. However, asynchronous C/R brings about many new challenges. For starters, asynchronous C/R introduces complex resource contention between the application and the C/R implementation. Additionally, some implementations adopt file-per-process writing strategies, which overwhelm PFS' at high core counts. In this work, we explore how multi-threaded POSIX I/O impacts aggregated throughput. To this extent we characterize the influence of different I/O parameters, such as the number of writer threads and how they access storage devices, has on aggregated I/O. We use the information gathered in this study to identify best practices when performing aggregated I/O as a first step in designing an efficient I/O aggregation scheme for asynchronous C/R.
Fichier principal
Vignette du fichier
Multithreaded_Checkpoint_Aggregation_Motivation.pdf (268.42 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY - Paternité

Dates et versions

hal-04343661 , version 1 (13-12-2023)

Licence

Paternité

Identifiants

Citer

Mikaila Gossman, Bogdan Nicolae, Jon Calhoun. Modeling Multi-Threaded Aggregated I/O for Asynchronous Checkpointing on HPC Systems. ISPDC 2023: The 22nd International Symposium on Parallel and Distributed Computing, Jul 2023, Bucharest, Romania. pp.101-105, ⟨10.1109/ISPDC59212.2023.00021⟩. ⟨hal-04343661⟩
5 Consultations
11 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More