Minimizing I/Os in Out-of-Core Task Tree Scheduling - Archive ouverte HAL
Article Dans Une Revue International Journal of Foundations of Computer Science Année : 2023

Minimizing I/Os in Out-of-Core Task Tree Scheduling

Résumé

Scientific applications are usually described using directed acyclic graphs, where nodes represent tasks and edges represent dependencies between tasks. For some applications, this graph is a tree: each task produces a single result used solely by its parent. The temporary results of each task have to be stored between their production and their use. We focus on the case when the data manipulated are very large. Then, during an execution, all data may not fit together in memory. In such a case, some data have to be temporarily written to disk and evicted from memory. These data are later read from disk when they are needed for computation. These Input/Output operations are very expensive; hence, our goal is to minimize their total volume. The order in which the tasks are processed considerably influences the amount of such Input/Output operations. Finding the schedule which minimizes this amount is an open problem that we revisit in this paper. We first formalize and generalize known results, and prove that existing solutions can be arbitrarily worse than the optimal. We then present an Integer Linear Program to solve it optimally. Finally, we propose a novel heuristic algorithm. We demonstrate its good performance through simulations on both synthetic and realistic trees built from actual scientific applications.

Domaines

Informatique [cs]
Fichier principal
Vignette du fichier
ijfcs.pdf (580.57 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

INSPIRE HEP  : Connectez-vous pour contacter le contributeur

https://hal.science/hal-03758021

Soumis le : mardi 13 septembre 2022-10:03:50

Dernière modification le : mercredi 13 novembre 2024-16:28:02

Archivage à long terme le : mercredi 14 décembre 2022-18:10:30

Télécharger pour visualiser

Dates et versions

hal-03758021 , version 1 (13-09-2022)

Licence

Paternité

Identifiants

Citer

Loris Marchal, Samuel McCauley, Bertrand Simon, Frédéric Vivien. Minimizing I/Os in Out-of-Core Task Tree Scheduling. International Journal of Foundations of Computer Science, 2023, 34 (01), pp.51-80. ⟨10.1142/s0129054122500186⟩. ⟨hal-03758021⟩

Exporter

BibTeX XML-TEI Dublin Core DC Terms EndNote DataCite

Collections

IN2P3 ENS-LYON CNRS INRIA UNIV-LYON1 INRIA2 UDL CC-IN2P3
117 Consultations
118 Téléchargements

Altmetric

Partager

More