Journal Articles International Journal of Foundations of Computer Science Year : 2023

Minimizing I/Os in Out-of-Core Task Tree Scheduling

Abstract

Scientific applications are usually described using directed acyclic graphs, where nodes represent tasks and edges represent dependencies between tasks. For some applications, this graph is a tree: each task produces a single result used solely by its parent. The temporary results of each task have to be stored between their production and their use. We focus on the case when the data manipulated are very large. Then, during an execution, all data may not fit together in memory. In such a case, some data have to be temporarily written to disk and evicted from memory. These data are later read from disk when they are needed for computation. These Input/Output operations are very expensive; hence, our goal is to minimize their total volume. The order in which the tasks are processed considerably influences the amount of such Input/Output operations. Finding the schedule which minimizes this amount is an open problem that we revisit in this paper. We first formalize and generalize known results, and prove that existing solutions can be arbitrarily worse than the optimal. We then present an Integer Linear Program to solve it optimally. Finally, we propose a novel heuristic algorithm. We demonstrate its good performance through simulations on both synthetic and realistic trees built from actual scientific applications.

Domains

Computer Science [cs]
Fichier principal
Vignette du fichier
ijfcs.pdf (580.57 Ko) Télécharger le fichier
Origin Files produced by the author(s)

INSPIRE HEP  : Connect in order to contact the contributor

https://hal.science/hal-03758021

Submitted on : Tuesday, September 13, 2022-10:03:50 AM

Last modification on : Wednesday, November 13, 2024-4:28:02 PM

Long-term archiving on : Wednesday, December 14, 2022-6:10:30 PM

Download

Dates and versions

hal-03758021 , version 1 (13-09-2022)

Licence

Attribution

Identifiers

Cite

Loris Marchal, Samuel McCauley, Bertrand Simon, Frédéric Vivien. Minimizing I/Os in Out-of-Core Task Tree Scheduling. International Journal of Foundations of Computer Science, 2023, 34 (01), pp.51-80. ⟨10.1142/s0129054122500186⟩. ⟨hal-03758021⟩

Export

BibTeX XML-TEI Dublin Core DC Terms EndNote DataCite

Collections

IN2P3 ENS-LYON CNRS INRIA UNIV-LYON1 INRIA2 UDL CC-IN2P3
124 View
127 Download

Altmetric

Share

More