Arbitration Policies for On-Demand User-Level I/O Forwarding on HPC Platforms - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Arbitration Policies for On-Demand User-Level I/O Forwarding on HPC Platforms

Résumé

I/O forwarding is a well-established and widelyadopted technique in HPC to reduce contention in the access to storage servers and transparently improve I/O performance. Rather than having applications directly accessing the shared parallel file system, the forwarding technique defines a set of I/O nodes responsible for receiving application requests and forwarding them to the file system, thus reshaping the flow of requests. The typical approach is to statically assign I/O nodes to applications depending on the number of compute nodes they use, which is not always necessarily related to their I/O requirements. Thus, this approach leads to inefficient usage of these resources. This paper investigates arbitration policies based on the applications I/O demands, represented by their access patterns. We propose a policy based on the Multiple-Choice Knapsack problem that seeks to maximize global bandwidth by giving more I/O nodes to applications that will benefit the most. Furthermore, we propose a userlevel I/O forwarding solution as an on-demand service capable of applying different allocation policies at runtime for machines where this layer is not present. We demonstrate our approach's applicability through extensive experimentation and show it can transparently improve global I/O bandwidth by up to 85% in a live setup compared to the default static policy.
Fichier principal
Vignette du fichier
Bezetal_Arbitration.pdf (525.75 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03149582 , version 1 (23-02-2021)

Identifiants

  • HAL Id : hal-03149582 , version 1

Citer

Jean Luca Bez, Alberto Miranda, Ramon Nou, Francieli Zanon Boito, Toni Cortes, et al.. Arbitration Policies for On-Demand User-Level I/O Forwarding on HPC Platforms. IPDPS 2021 - 35th IEEE International Parallel and Distributed Processing Symposium, May 2021, Portland, Oregon / Virtual, United States. ⟨hal-03149582⟩
194 Consultations
308 Téléchargements

Partager

Gmail Facebook X LinkedIn More