Communication Dans Un Congrès Année : 2025

Erasure Coding Aware Block Placement for Data-Intensive Applications

Résumé

Erasure Coding (EC) has recently been integrated and deployed in the Hadoop Distributed File System (HDFS) to provide the same fault tolerance guarantees as replication, but with significantly less storage overhead. When EC is used, data reads typically involve only data chunks. In this paper, we study the effect of data chunk distribution on the performance of reads and data-intensive applications, and present the design and evaluation of an erasure coding aware (EC-aware) block placement that balances the distribution of data chunks across nodes. Experimental results show that EC-aware block placement can reduce the execution time of Sort and WordCount applications by up to 25%.

Fichier principal
Vignette du fichier
3719330.3721229.pdf (911.72 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
Licence

Dates et versions

hal-05016741 , version 1 (05-11-2025)

Licence

Identifiants

Citer

Shadi Ibrahim, Jad Darrous. Erasure Coding Aware Block Placement for Data-Intensive Applications. CHEOPS 2025 - 5th Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems, Mar 2025, Rotterdam Netherlands, Netherlands. pp.15-22, ⟨10.1145/3719330.3721229⟩. ⟨hal-05016741⟩
401 Consultations
155 Téléchargements

Altmetric

Partager

  • More