Online Tuning of EASY-Backfilling using Queue Reordering Policies - Archive ouverte HAL Access content directly
Journal Articles IEEE Transactions on Parallel and Distributed Systems Year : 2018

Online Tuning of EASY-Backfilling using Queue Reordering Policies

Abstract

The EASY-FCFS heuristic is the basic building block of job scheduling policies in most parallel High Performance Computing platforms. Despite its simplicity, and the guarantee of no job starvation, it could still be improved on a per-system basis. Such tuning is difficult because of non-linearities in the scheduling process. The study conducted in this paper considers an online approach to the automatic tuning of the EASY heuristic for HPC platforms. More precisely, we consider the problem of selecting a reordering policy for the job queue under several feedback modes. We show via a comprehensive experimental validation on actual logs that periodic simulation of historical data can be used to recover existing in-hindsight results that allow to divide the average waiting time by almost 2. This results holds even when the simulator results are noisy. Moreover, we show that good performances can still be obtained without a simulator, under what is called bandit feedback - when we can only observe the performance of the algorithm that was picked on the live system. Indeed, a simple multi-armed bandit algorithm can reduce the average waiting time by 40 percent.
No file

Dates and versions

hal-01963216 , version 1 (21-12-2018)

Identifiers

Cite

Éric Gaussier, Jérôme Lelong, Valentin Reis, Denis Trystram. Online Tuning of EASY-Backfilling using Queue Reordering Policies. IEEE Transactions on Parallel and Distributed Systems, 2018, 29 (10), pp.2304-2316. ⟨10.1109/TPDS.2018.2820699⟩. ⟨hal-01963216⟩
141 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More