Benchmarking Hadoop Performance in the Cloud - An in Depth Study of Resource Management and Energy Consumption - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Benchmarking Hadoop Performance in the Cloud - An in Depth Study of Resource Management and Energy Consumption

Résumé

Virtual technologies have proven their capabilities to ensure good performance in the context of high performance computing (HPC). During the last decade, the big data tools have been emerging, they have their own needs in performance and infrastructure. Having a wide breadth of experience in the HPC domain, the experts can evaluate the infrastructures used to run big data tools easily. The outcome of this paper is the evaluation of two technologies of virtualization in the context of big data tools. We compare the performance and the energy consumption of two technologies of virtualization (Docker containers and VMware) and benchmark the software Hadoop (JoshBaer, 2015) using these environments. Firstly, the aim is the reduction of the Hadoop deployment cost using the cloud. Secondly, we discuss and analyze the assumptions learned from the HPC experiments and their applicability in the big data context. Thirdly, the Hadoop community finds an in-depth study of the resource consumption depending on the deployment environment. We come to the point that the use of the Docker container gives better performance in most experiments. Besides, the energy consumption varies according to the executed workload.
Fichier principal
Vignette du fichier
Closer2016-3.pdf (691.14 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01325027 , version 1 (01-06-2016)

Identifiants

Citer

Aymen Jlassi, Patrick Martineau. Benchmarking Hadoop Performance in the Cloud - An in Depth Study of Resource Management and Energy Consumption. The 6th International Conference on Cloud Computing and Services Science, Apr 2016, ROME, Italy. ⟨10.5220/0005861701920201⟩. ⟨hal-01325027⟩
221 Consultations
927 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More