Adaptive Modelling and Control in Distributed Systems - Archive ouverte HAL
Communication Dans Un Congrès Année : 2015

Adaptive Modelling and Control in Distributed Systems

Résumé

Companies have growing amounts of data to store and to process. In response to these new processing challenges, Google developed MapReduce, a parallel programming paradigm which is becoming the major tool for BigData treatment. Even if MapReduce is used by most IT companies, ensuring its performances while minimizing costs is a real challenge requiring a high level of expertise. Modelling and control of MapReduce have been developed in the last years, however there are still many problems caused by the software's high variability. To tackle the latter issue, this paper proposes an on-line model estimation algorithm for MapReduce systems. An adaptive control strategy is developed and implemented to guarantee response time performances under a concurrent workload while minimizing resource use. Results have been validated using a 40 nodes MapReduce cluster under a data intensive Business Intelligence workload running on Grid5000, a French national cloud. The experiments show that the adaptive control algorithm manages to guarantee performances and low costs even in a highly variable environment.
Fichier principal
Vignette du fichier
SCERF_PhdForumSRDS2015.pdf (210.02 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01240558 , version 1 (09-12-2015)

Identifiants

  • HAL Id : hal-01240558 , version 1

Citer

Sophie Cerf, Mihaly Berekmeri, Nicolas Marchand, Sara Bouchenak, Bogdan Robu. Adaptive Modelling and Control in Distributed Systems. SRDS 2015 - Phd Forum 34th International Symposium on Reliable Distributed Systems, McGill University, Sep 2015, Montreal, Canada. ⟨hal-01240558⟩
980 Consultations
293 Téléchargements

Partager

More