A Multi-Metric Adaptive Stream Processing System - Archive ouverte HAL Access content directly
Conference Papers Year :

A Multi-Metric Adaptive Stream Processing System

Daniel Wladdimiro
Luciana Arantes
Pierre Sens
Nicolas Hidalgo
  • Function : Author
  • PersonId : 1035470

Abstract

Stream processing systems (SPS) have to deal with highly dynamic scenarios where its adaptation is mandatory in order to accomplish realistic applications requirements. In this work, we propose a new adaptive SPS for real-time processing that, based on input data rate variation, dynamically adapts the number of active operator replicas. Our SPS extends Storm by pre-allocating, for each operator, a set of inactive replicas which are activated (or deactivated) when necessary without the Storm reconfiguration cost. We exploit the MAPE model and define a new metric that aggregates the value of multiple metrics to dynamically changes the number of replicas of an operator. We deploy our SPS over Google Cloud Platform and results confirm that our metric can tolerate highly dynamic conditions, improving resource usage while preserving high throughput and low latency.
Fichier principal
Vignette du fichier
Paper___A_Multi_Metric_Adaptive_for_Stream_Processing_System___NCA2021 (4).pdf (491.66 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03516376 , version 1 (07-01-2022)

Identifiers

  • HAL Id : hal-03516376 , version 1

Cite

Daniel Wladdimiro, Luciana Arantes, Pierre Sens, Nicolas Hidalgo. A Multi-Metric Adaptive Stream Processing System. NCA 2021 - 20th IEEE International Symposium on Network Computing and Applications, Nov 2021, Cambridge, Boston, United States. ⟨hal-03516376⟩
67 View
115 Download

Share

Gmail Facebook Twitter LinkedIn More