Implementing and optimizing multiple group-by query in a MapReduce approach - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Algorithms and Computational Technology Année : 2010

Implementing and optimizing multiple group-by query in a MapReduce approach

J. Pan
  • Fonction : Auteur
F. Magoules
Y. Le Biannic
  • Fonction : Auteur

Résumé

MapReduce model is a new parallel programming model initially developed for large-scale web content processing. Data analysis meets the issue of how to do calculation over extremely large dataset. The arrival of MapReduce provides a chance to utilize commodity hardware for massively parallel data analysis applications. The translation and optimization from relational algebra operators to MapReduce programs is still an open and dynamic research field. In this paper, we focus on a special type of data analysis query, namely, multiple group by query. We first study the communication cost of MapReduce model, then we give an initial implementation of multiple group by query. We then propose an optimized version which addresses and improves the communication cost issues. Our optimized version shows a better accelerating ability and a better scalability than the other version.

Dates et versions

hal-00828545 , version 1 (31-05-2013)

Identifiants

Citer

J. Pan, F. Magoules, Y. Le Biannic. Implementing and optimizing multiple group-by query in a MapReduce approach. Journal of Algorithms and Computational Technology, 2010, 4 (2), pp.183-206. ⟨10.1260/1748-3018.4.2.183⟩. ⟨hal-00828545⟩
41 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More