Seamlessly Scaling Applications with DAPHNE
Résumé
When developing scientific and/or data science applications, users might start by using a scripting language to quickly produce a prototype, but eventually need to rewrite their code in a higher performance language when performance becomes a bottleneck.
A similar situation arises when the applications need to scale beyond a single node. Users need to consequently
adapt their code to integrate domain decomposition, synchronization, and communication libraries such as MPI.
In this paper, we present ongoing work on the distributed scheduler of DAPHNE (https://daphne-eu.eu), an infrastructure for optimizing data analysis pipelines.
With DAPHNE, users can write their applications once in a high level language, as they would
in Python or Julia, and benefit from seamless scalability across computing nodes.
Origine | Fichiers produits par l'(les) auteur(s) |
---|