A Two-Level Formal Model for Big Data Processing Programs - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Science of Computer Programming Année : 2021

A Two-Level Formal Model for Big Data Processing Programs

Résumé

This paper proposes a model for specifying data flow-based parallel data processing programs agnostic of target Big Data processing frameworks. The paper focuses on the formal abstract specification of non-iterative and iterative programs, generalizing the strategies adopted by data flow Big Data processing frameworks. The proposed model relies on Monoid Algebra and Petri Nets to abstract Big Data processing programs in two levels: a higher level representing the program data flow and a lower level representing data transformation operations (e.g., filtering, aggregation, join). We extend the model for data processing programs, for modeling iterative data processing programs. The general specification of these programs implemented by data flow-based parallel programming models is essential given the democratization of iterative and greedy Big Data analytics algorithms. Indeed, these algorithms call for revisiting parallel programming models to express iterations. The paper gives a comparative analysis of the iteration strategies proposed by Apache Spark, DryadLINQ, Apache Beam, and Apache Flink. It discusses how the model achieves to generalize these strategies.
Fichier principal
Vignette du fichier
Modeling_Big_Data_Processing_Programs___SBMF___Extended___Second_Version.pdf (650.78 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03494461 , version 1 (19-12-2021)

Identifiants

Citer

João Batista de Souza Neto, Anamaria Martins Moreira, Genoveva Vargas-Solar, Martin A Musicante. A Two-Level Formal Model for Big Data Processing Programs. Science of Computer Programming, 2021, 215 (1), ⟨10.1016/j.scico.2021.102764⟩. ⟨hal-03494461⟩
18 Consultations
218 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More