Cost-efficient big intermediate data placement in a collaborative cloud storage environment - Archive ouverte HAL
Communication Dans Un Congrès Année : 2017

Cost-efficient big intermediate data placement in a collaborative cloud storage environment

Résumé

Collaborative cloud storage environment, which share resources of multiple geographically distributed datacenters owned by different providers enable scientific workflow from different locations to process large scale big intermediate data through the Internet. Distributed datacenters are federated and each member can collaborate with each other to efficiently share and process the intermediate data from distributed workflow instances. This paper focuses on the storage cost minimization of intermediate data placement in federated cloud datacenters. Through collaborative and federation mechanisms, we propose an exact federation data placement algorithm based on integer linear programming model (ILP) to assist multiple datacenters hosting intermediate data files generated from a scientific workflow. Under the constraints of the problem, the proposed algorithm finds an optimal intermediate data placement with a cost saving over the federated cloud datacenters, taking into account scientific user requirements, data dependency and size. Experimental results show the cost-efficiency of the proposed cloud storage federation algorithm.
Fichier principal
Vignette du fichier
2017 IEEE 19th International Conference_Cost-Efficient Big Intermediate Data Placement in a.pdf (795.94 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04390441 , version 1 (13-02-2024)

Identifiants

Citer

Sonia Ikken, Eric Renault, Amine Barkat, Abdelkamel Tari, Tahar Kechadi. Cost-efficient big intermediate data placement in a collaborative cloud storage environment. 2017 IEEE 19th International Conference on High Performance Computing and Communications, IEEE 15th International Conference on Smart City and IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Dec 2017, Bangkok, Thailand. pp.514-521, ⟨10.1109/HPCC-SmartCity-DSS.2017.67⟩. ⟨hal-04390441⟩
93 Consultations
24 Téléchargements

Altmetric

Partager

More