Shared-Memory Communication for Containerized Workflows - Joint Laboratory on Extreme Scale Computing
Communication Dans Un Congrès Année : 2021

Shared-Memory Communication for Containerized Workflows

Tanner Hobson
  • Fonction : Auteur
  • PersonId : 1096317
Orcun Yildiz
  • Fonction : Auteur
  • PersonId : 1029719
Bogdan Nicolae
Jian Huang
  • Fonction : Auteur
  • PersonId : 1096318
Tom Peterka
  • Fonction : Auteur
  • PersonId : 1096319

Résumé

Scientific computation increasingly consists of a workflow of interrelated tasks. Containerization can make workflow systems more manageable, reproducible, and portable, but containers can impede communication due to their focus on encapsulation. In some circumstances, shared-memory regions are an effective way to improve performance of workflows; however sharing memory between containerized workflow tasks is difficult. In this work, we have created a software library called Dhmem that manages shared memory between workflow tasks in separate containers, with minimal code change and performance overhead. Instead of all code being in the same container, Dhmem allows a separate container for each workflow task to be constructed completely independently. Dhmem enables additional functionality: easy integration in existing workflow systems, communication configuration at runtime based on the environment, and scalable performance.
Fichier principal
Vignette du fichier
Shared-Memory_Communication_for_Containerized_Workflows.pdf (441.14 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03200931 , version 1 (17-04-2021)

Identifiants

  • HAL Id : hal-03200931 , version 1

Citer

Tanner Hobson, Orcun Yildiz, Bogdan Nicolae, Jian Huang, Tom Peterka. Shared-Memory Communication for Containerized Workflows. CCGrid'21: The 21th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, May 2021, Melbourne, Australia. ⟨hal-03200931⟩

Collections

JLESC
121 Consultations
1209 Téléchargements

Partager

More