Salamander: a Holistic Scheduling of MapReduce Jobs on Ephemeral Cloud Resources - Archive ouverte HAL Access content directly
Conference Papers Year : 2020

Salamander: a Holistic Scheduling of MapReduce Jobs on Ephemeral Cloud Resources

Abstract

Most cloud data centers are over-provisioned and underutilized, primarily to handle peak loads and sudden failures. This has motivated many researchers to reclaim the unused resources, which are by nature ephemeral, to run data-intensive applications at a lower cost. Hadoop MapReduce is one of those applications. However, it was designed on the assumption that resources are available as long as users pay for the service. In order to make it possible for Hadoop to run on unused (ephemeral) resources, we have designed a heterogeneity and volatility-aware holistic scheduler consisting of three different components: (1) A MapReduce task and job scheduler that relies on a global vision of resource utilization predictions, (2) a scheduler-based data placement strategy that improves the data locality, and (3) a reactive QoS controller that ensures customers’ service-level agreement (SLA) and minimizes interference between co-located workloads. Our framework makes it possible to take advantage of ephemeral resources efficiently. Indeed, for a given set of jobs, it reduces the overall execution time by up to 47.6% and an average of 18.7% as compared to state-of-the-art strategies.
Fichier principal
Vignette du fichier
PID6379855.pdf (579.67 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02497029 , version 1 (03-03-2020)

Identifiers

  • HAL Id : hal-02497029 , version 1

Cite

Mohamed Handaoui, Jean-Emile Dartois, Laurent Lemarchand, Jalil Boukhobza. Salamander: a Holistic Scheduling of MapReduce Jobs on Ephemeral Cloud Resources. CCGRID 2020 - 20th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, Nov 2020, Melbourne, Australia. pp.1-10. ⟨hal-02497029⟩
246 View
302 Download

Share

Gmail Facebook X LinkedIn More