ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Résumé

Modern scientific workflows require hybrid infrastructures combining numerous decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the Computing Continuum) to enable their optimized execution. Understanding and optimizing the performance of such complex Edge-to-Cloud workflows is challenging. Capturing the provenance of key performance indicators, with their related data and processes, may assist in understanding and optimizing workflow executions. However, the capture overhead can be prohibitive, particularly in resource-constrained devices, such as the ones on the IoT/Edge. To address this challenge, based on a performance analysis of existing systems, we propose ProvLight, a tool to enable efficient provenance capture on the IoT/Edge. We leverage simplified data models, data compression and grouping, and lightweight transmission protocols to reduce overheads. We further integrate ProvLight into the E2Clab framework to enable workflow provenance capture across the Edge-to-Cloud Continuum. This integration makes E2Clab a promising platform for the performance optimization of applications through reproducible experiments. We validate ProvLight at a large scale with synthetic workloads on 64 real-life IoT/Edge devices in the FIT IoT LAB testbed. Evaluations show that ProvLight outperforms state-of-the-art systems like ProvLake and DfAnalyzer in resource-constrained devices. ProvLight is 26---37x faster to capture and transmit provenance data; uses 5---7x less CPU; 2x less memory; transmits 2x less data; and consumes 2---2.5x less energy. ProvLight and E2Clab are available as open-source tools.
Fichier principal
Vignette du fichier
0_main.pdf (1.5 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04161546 , version 1 (18-07-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04161546 , version 1

Citer

Daniel Rosendo, Marta Mattoso, Alexandru Costan, Renan Souza, Débora Pina, et al.. ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum. IEEE Cluster 2023 - IEEE International Conference on Cluster Computing, Oct 2023, Santa Fe, New Mexico, United States. pp.1-13. ⟨hal-04161546⟩
83 Consultations
37 Téléchargements

Partager

Gmail Facebook X LinkedIn More