BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map-Reduce Applications - Archive ouverte HAL Access content directly
Conference Papers Year : 2010

BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map-Reduce Applications

(1) , (1) , (1) , (1) , (1)
1
Bogdan Nicolae
  • Function : Author
  • PersonId : 862774
Diana Moise
  • Function : Author
  • PersonId : 867268
Luc Bougé
Matthieu Dorier
  • Function : Author
  • PersonId : 865414

Abstract

Hadoop is a software framework supporting the Map-Reduce programming model. It relies on the Hadoop Distributed File System (HDFS) as its primary storage system. The efficiency of HDFS is crucial for the performance of Map-Reduce applications. We substitute the original HDFS layer of Hadoop with a new, concurrency-optimized data storage layer based on the BlobSeer data management service. Thereby, the efficiency of Hadoop is significantly improved for data-intensive Map-Reduce applications, which naturally exhibit a high degree of data access concurrency. Moreover, BlobSeer's features (built-in versioning, its support for concurrent append operations) open the possibility for Hadoop to further extend its functionalities. We report on extensive experiments conducted on the Grid'5000 testbed. The results illustrate the benefits of our approach over the original HDFS-based implementation of Hadoop.
Fichier principal
Vignette du fichier
PID1126243.pdf (181.61 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

inria-00456801 , version 1 (15-02-2010)

Identifiers

Cite

Bogdan Nicolae, Diana Moise, Gabriel Antoniu, Luc Bougé, Matthieu Dorier. BlobSeer: Bringing High Throughput under Heavy Concurrency to Hadoop Map-Reduce Applications. 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2010), IEEE and ACM, Apr 2010, Atlanta, United States. ⟨10.1109/IPDPS.2010.5470433⟩. ⟨inria-00456801⟩
374 View
600 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More