Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley - Archive ouverte HAL
Article Dans Une Revue Future Generation Computer Systems Année : 2018

Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley

Résumé

Burst Buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications – which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications.

Dates et versions

hal-01742090 , version 1 (23-03-2018)

Identifiants

Citer

Orcun Yildiz, Amelie Chi Zhou, Shadi Ibrahim. Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley. Future Generation Computer Systems, 2018, 86, pp.308-318. ⟨10.1016/j.future.2018.03.029⟩. ⟨hal-01742090⟩
430 Consultations
0 Téléchargements

Altmetric

Partager

More