Putting Data Science Pipelines on the Edge
Abstract
This paper proposes a composable "Just in Time Architecture" for Data Science (DS) Pipelines named JITA-4DS and associated resource management techniques for configuring disaggregated data centers (DCs). DCs under our approach are composable based on vertical integration of the application, middleware/operating system, and hardware layers customized dynamically to meet application Service Level Objectives (SLO-application-aware management). Thereby, pipelines utilize a set of flexible building blocks that can be dynamically and automatically assembled and reassembled to meet the dynamic changes in the workload's SLOs. To assess disaggregated DC's, we study how to model and validate their performance in large-scale settings.
Domains
Databases [cs.DB]Origin | Files produced by the author(s) |
---|