A Control Approach for Performance of Big Data Systems - Archive ouverte HAL
Communication Dans Un Congrès Année : 2014

A Control Approach for Performance of Big Data Systems

Résumé

We are at the dawn of a huge data explosion therefore companies have fast growing amounts of data to process. For this purpose Google developed MapReduce, a parallel programming paradigm which is slowly becoming the de facto tool for Big Data analytics. Although to some extent its use is already wide-spread in the industry, ensuring performance constraints for such a complex system poses great challenges and its management requires a high level of expertise. This paper answers these challenges by providing the first autonomous controller that ensures service time constraints of a concurrent MapReduce workload. We develop the first dynamic model of a MapReduce cluster. Furthermore, PI feedback control is developed and implemented to ensure service time constraints. A feedforward controller is added to improve control response in the presence of disturbances, namely changes in the number of clients. The approach is validated online on a real 40 node MapReduce cluster, running a data intensive Business Intelligence workload. Our experiments demonstrate that the designed control is successful in assuring service time constraints.
Fichier principal
Vignette du fichier
ifacconf.pdf (624.84 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-00980372 , version 1 (05-01-2015)

Identifiants

Citer

Mihaly Berekmeri, Damián Serrano, Sara Bouchenak, Nicolas Marchand, Bogdan Robu. A Control Approach for Performance of Big Data Systems. IFAC WC 2014 - 19th IFAC World Congress, Aug 2014, Le Cap, South Africa. ⟨10.3182/20140824-6-ZA-1003.01319⟩. ⟨hal-00980372⟩
571 Consultations
1096 Téléchargements

Altmetric

Partager

More