Decomposition and Sharing User-defined Aggregation: from Theory to Practice - Archive ouverte HAL
Pré-Publication, Document De Travail Année : 2018

Decomposition and Sharing User-defined Aggregation: from Theory to Practice

Résumé

We study the problems of decomposing and sharing user-defined aggregate functions in distributed and parallel computing. Aggre-gation usually needs to satisfy the distributive property to compute in parallel, and to leverage optimization in multidimensional data analysis and conjunctive query with aggregation. However, this property is too restricted to allow more aggregation to benefit from these advantages. We propose for user-defined aggregation functions a formal framework to relax the previous condition, and we map this framework to the MRC, an efficient computation model in MapReduce, to automatically generate efficient partial aggrega-tion functions. Moreover, we identify the complete conditions for sharing the result of practical user-defined aggregation without scanning base data, and propose a hybrid solution, the symbolic index, pull-up rules, to optimize the sharing process.
Fichier principal
Vignette du fichier
report_ZHANGChao.pdf (1015.07 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01877088 , version 1 (19-09-2018)
hal-01877088 , version 2 (18-10-2018)

Identifiants

  • HAL Id : hal-01877088 , version 2

Citer

Chao Zhang, Farouk Toumani. Decomposition and Sharing User-defined Aggregation: from Theory to Practice. 2018. ⟨hal-01877088v2⟩
134 Consultations
174 Téléchargements

Partager

More