A Control Approach for Performance of Big Data Systems - Archive ouverte HAL Access content directly
Conference Papers Year :

A Control Approach for Performance of Big Data Systems

(1, 2) , (2) , (2) , (1) , (1)


We are at the dawn of a huge data explosion therefore companies have fast growing amounts of data to process. For this purpose Google developed MapReduce, a parallel programming paradigm which is slowly becoming the de facto tool for Big Data analytics. Although to some extent its use is already wide-spread in the industry, ensuring performance constraints for such a complex system poses great challenges and its management requires a high level of expertise. This paper answers these challenges by providing the first autonomous controller that ensures service time constraints of a concurrent MapReduce workload. We develop the first dynamic model of a MapReduce cluster. Furthermore, PI feedback control is developed and implemented to ensure service time constraints. A feedforward controller is added to improve control response in the presence of disturbances, namely changes in the number of clients. The approach is validated online on a real 40 node MapReduce cluster, running a data intensive Business Intelligence workload. Our experiments demonstrate that the designed control is successful in assuring service time constraints.
Fichier principal
Vignette du fichier
ifacconf.pdf (624.84 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-00980372 , version 1 (05-01-2015)



Mihaly Berekmeri, Damián Serrano, Sara Bouchenak, Nicolas Marchand, Bogdan Robu. A Control Approach for Performance of Big Data Systems. IFAC WC 2014 - 19th IFAC World Congress, Aug 2014, Le Cap, South Africa. ⟨10.3182/20140824-6-ZA-1003.01319⟩. ⟨hal-00980372⟩
516 View
1057 Download



Gmail Facebook Twitter LinkedIn More