Building the I (Interoperability) of FAIR for Performance Reproducibility of Large-Scale Composable Workflows in RECUP - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Building the I (Interoperability) of FAIR for Performance Reproducibility of Large-Scale Composable Workflows in RECUP

Résumé

Scientific computing communities increasingly run their experiments using complex data-and compute-intensive workflows that utilize distributed and heterogeneous architectures targeting numerical simulations and machine learning, often executed on the Department of Energy Leadership Computing Facilities (LCFs). We argue that a principled, systematic approach to implementing FAIR principles at scale, including fine-grained metadata extraction and organization, can help with the numerous challenges to performance reproducibility posed by such workflows. We extract workflow patterns, propose a set of tools to manage the entire life cycle of performance metadata, and aggregate them in an HPC-ready framework for reproducibility (RECUP). We describe the challenges in making these tools interoperable, preliminary work, and lessons learned from this experiment.
Fichier principal
Vignette du fichier
RECUP_E_Science_2023_ReWords_workshop.pdf (586.8 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04343665 , version 1 (14-12-2023)

Licence

Paternité

Identifiants

Citer

Bogdan Nicolae, Tanzima Islam, Robert Ross, Huub van Dam, Kevin Assogba, et al.. Building the I (Interoperability) of FAIR for Performance Reproducibility of Large-Scale Composable Workflows in RECUP. e-Science 2023: The IEEE 19th International Conference on e-Science, Oct 2023, Limassol, Cyprus. pp.1-7, ⟨10.1109/e-Science58273.2023.10254808⟩. ⟨hal-04343665⟩
7 Consultations
27 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More