Adding topology and memory awareness in data aggregation algorithms - Archive ouverte HAL
Article Dans Une Revue Future Generation Computer Systems Année : 2024

Adding topology and memory awareness in data aggregation algorithms

Résumé

With the growing gap between computing power and the ability of large-scale systems to ingest data, I/O is becoming the bottleneck for many scientific applications. Improving read and write performance thus becomes decisive, and requires consideration of the complexity of architectures. In this paper, we introduce TAPIOCA, an architecture-aware data aggregation library. TAPIOCA offers an optimized implementation of the two-phase I/O scheme for collective I/O operations, taking advantage of the many levels of memory and storage that populate modern HPC systems, and leveraging network topology. We show that TAPIOCA can significantly improve the I/O bandwidth of synthetic benchmarks and I/O kernels of scientific applications running on leading supercomputers. For example, on HACC-IO, a cosmology code, TAPIOCA improves data writing by a factor of 13 on nearly a third of the target supercomputer.
Fichier principal
Vignette du fichier
FGCS_JLESC_Special_Issue-10.pdf (734.77 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04783379 , version 1 (14-11-2024)
hal-04783379 , version 2 (15-11-2024)

Licence

Identifiants

Citer

François Tessier, Venkatram Vishwanath, Emmanuel Jeannot. Adding topology and memory awareness in data aggregation algorithms. Future Generation Computer Systems, 2024, 159, pp.188-203. ⟨10.1016/j.future.2024.05.016⟩. ⟨hal-04783379v2⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

More