Multi-Level Elasticity for Data Stream Processing - Archive ouverte HAL Access content directly
Journal Articles IEEE Transactions on Parallel and Distributed Systems Year : 2019

Multi-Level Elasticity for Data Stream Processing

Abstract

This paper investigates reactive elasticity in stream processing environments where the performance goal is to analyze large amounts of data with low latency and minimum resources. Working in the context of Apache Storm, we propose an elastic management strategy which modulates the parallelism degree of applications' components while explicitly addressing the hierarchy of execution containers (virtual machines, processes and threads). We show that provisioning the wrong kind of container may lead to performance degradation and propose a solution that provisions the least expensive container (with minimum resources) to increase performance. We describe our monitoring metrics and show how we take into account the specifics of an execution environment. We provide an experimental evaluation with real-world applications which validates the applicability of our approach.
Not file

Dates and versions

hal-02143440 , version 1 (29-05-2019)

Identifiers

Cite

Vania Marangozova-Martin, Ahmed El Rheddane, Noel de Palma. Multi-Level Elasticity for Data Stream Processing. IEEE Transactions on Parallel and Distributed Systems, 2019, pp.1-12. ⟨10.1109/TPDS.2019.2907950⟩. ⟨hal-02143440⟩
79 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More