Decomposition and Sharing User-defined Aggregation: from Theory to Practice - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year : 2018

Decomposition and Sharing User-defined Aggregation: from Theory to Practice

Abstract

We study the problems of decomposing and sharing user-defined aggregate functions in distributed and parallel computing. Aggre-gation usually needs to satisfy the distributive property to compute in parallel, and to leverage optimization in multidimensional data analysis and conjunctive query with aggregation. However, this property is too restricted to allow more aggregation to benefit from these advantages. We propose for user-defined aggregation functions a formal framework to relax the previous condition, and we map this framework to the MRC, an efficient computation model in MapReduce, to automatically generate efficient partial aggrega-tion functions. Moreover, we identify the complete conditions for sharing the result of practical user-defined aggregation without scanning base data, and propose a hybrid solution, the symbolic index, pull-up rules, to optimize the sharing process.
Fichier principal
Vignette du fichier
report_ZHANGChao.pdf (1015.07 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01877088 , version 1 (19-09-2018)
hal-01877088 , version 2 (18-10-2018)

Identifiers

  • HAL Id : hal-01877088 , version 2

Cite

Chao Zhang, Farouk Toumani. Decomposition and Sharing User-defined Aggregation: from Theory to Practice. 2018. ⟨hal-01877088v2⟩
120 View
157 Download

Share

Gmail Facebook X LinkedIn More