Feedback Autonomic Provisioning for Guaranteeing Performance in MapReduce Systems - Archive ouverte HAL
Article Dans Une Revue IEEE Transactions on Cloud Computing Année : 2016

Feedback Autonomic Provisioning for Guaranteeing Performance in MapReduce Systems

Résumé

Companies have a fast growing amounts of data to process and store, a data explosion is happening next to us. Currently one of the most common approaches to treat these vast data quantities are based on the MapReduce parallel programming paradigm. While its use is widespread in the industry, ensuring performance constraints, while at the same time minimizing costs, still provides considerable challenges. We propose a coarse grained control theoretical approach, based on techniques that have already proved their usefulness in the control community. We introduce the first algorithm to create dynamic models for Big Data MapReduce systems, running a concurrent workload. Furthermore we identify two important control use cases: relaxed performance - minimal resource and strict performance. For the first case we develop two feedback control mechanism. A classical feedback controller and an evenbased feedback, that minimises the number of cluster reconfigurations as well. Moreover, to address strict performance requirements a feedforward predictive controller that efficiently suppresses the effects of large workload size variations is developed. All the controllers are validated online in a benchmark running in a real 60 node MapReduce cluster, using a data intensive Business Intelligence workload. Our experiments demonstrate the success of the control strategies employed in assuring service time constraints.
Fichier principal
Vignette du fichier
IEEE TCC 2014.pdf (2.11 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01297026 , version 1 (01-04-2016)
hal-01297026 , version 2 (02-06-2016)

Identifiants

Citer

Mihaly Berekmeri, Damián Serrano, Sara Bouchenak, Nicolas Marchand, Bogdan Robu. Feedback Autonomic Provisioning for Guaranteeing Performance in MapReduce Systems. IEEE Transactions on Cloud Computing, 2016, ⟨10.1109/TCC.2016.2550047⟩. ⟨hal-01297026v1⟩
1070 Consultations
470 Téléchargements

Altmetric

Partager

More