A Multi-Metric Adaptive Stream Processing System - Archive ouverte HAL
Communication Dans Un Congrès Année : 2021

A Multi-Metric Adaptive Stream Processing System

Daniel Wladdimiro
Luciana Arantes
Pierre Sens
Nicolas Hidalgo
  • Fonction : Auteur
  • PersonId : 1035470

Résumé

Stream processing systems (SPS) have to deal with highly dynamic scenarios where its adaptation is mandatory in order to accomplish realistic applications requirements. In this work, we propose a new adaptive SPS for real-time processing that, based on input data rate variation, dynamically adapts the number of active operator replicas. Our SPS extends Storm by pre-allocating, for each operator, a set of inactive replicas which are activated (or deactivated) when necessary without the Storm reconfiguration cost. We exploit the MAPE model and define a new metric that aggregates the value of multiple metrics to dynamically changes the number of replicas of an operator. We deploy our SPS over Google Cloud Platform and results confirm that our metric can tolerate highly dynamic conditions, improving resource usage while preserving high throughput and low latency.
Fichier principal
Vignette du fichier
Paper___A_Multi_Metric_Adaptive_for_Stream_Processing_System___NCA2021 (4).pdf (491.66 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03516376 , version 1 (07-01-2022)

Identifiants

  • HAL Id : hal-03516376 , version 1

Citer

Daniel Wladdimiro, Luciana Arantes, Pierre Sens, Nicolas Hidalgo. A Multi-Metric Adaptive Stream Processing System. NCA 2021 - 20th IEEE International Symposium on Network Computing and Applications, Nov 2021, Cambridge, Boston, United States. ⟨hal-03516376⟩
133 Consultations
190 Téléchargements

Partager

More