Tuning EASY-Backfilling Queues - Archive ouverte HAL Access content directly
Conference Papers Year : 2018

Tuning EASY-Backfilling Queues

Abstract

EASY-Backfilling is a popular scheduling heuristic for allocating jobs in large scale High Performance Computing platforms. While its aggressive reservation mechanism is fast and prevents job starvation, it does not try to optimize any scheduling objective per se. We consider in this work the problem of tuning EASY using queue reordering policies. More precisely, we propose to tune the reordering using a simulation-based methodology. For a given system, we choose the policy in order to minimize the average waiting time. This methodology departs from the First-Come, First-Serve rule and introduces a risk on the maximum values of the waiting time, which we control using a queue thresholding mechanism. This new approach is evaluated through a comprehensive experimental campaign on five production logs. In particular, we show that the behavior of the systems under study is stable enough to learn a heuristic that generalizes in a train/test fashion. Indeed, the average waiting time can be reduced consistently (between 11% to 42% for the logs used) compared to EASY, with almost no increase in maximum waiting times. This work departs from previous learning-based approaches and shows that scheduling heuristics for HPC can be learned directly in a policy space.
Fichier principal
Vignette du fichier
paper.pdf (712.2 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01522459 , version 1 (15-05-2017)

Identifiers

Cite

Jérôme Lelong, Valentin Reis, Denis Trystram. Tuning EASY-Backfilling Queues. 21st Workshop on Job Scheduling Strategies for Parallel Processing, May 2017, Orlando, United States. pp.43-61, ⟨10.1007/978-3-319-77398-8_3⟩. ⟨hal-01522459⟩
451 View
1163 Download

Altmetric

Share

Gmail Facebook X LinkedIn More