Towards inference delivery networks: distributing machine learning with optimality guarantees - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Towards inference delivery networks: distributing machine learning with optimality guarantees

Résumé

We present the novel idea of inference delivery networks (IDN), networks of computing nodes that coordinate to satisfy inference requests achieving the best trade-off between latency and accuracy. IDNs bridge the dichotomy between device and cloud execution by integrating inference delivery at the various tiers of the infrastructure continuum (access, edge, regional data center, cloud). We propose a distributed dynamic policy for ML model allocation in an IDN by which each node periodically updates its local set of inference models based on requests observed during the recent past plus limited information exchange with its neighbor nodes. Our policy offers strong performance guarantees in an adversarial setting and shows improvements over greedy heuristics with similar complexity in realistic scenarios.
Fichier principal
Vignette du fichier
sisalem21medcomnet.pdf (517.87 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03376168 , version 1 (13-10-2021)

Identifiants

Citer

Tareq Si Salem, Gabriele Castellano, Giovanni Neglia, Fabio Pianese, Andrea Araldo. Towards inference delivery networks: distributing machine learning with optimality guarantees. MEDCOMNET 2021 - 19th Mediterranean Communication and Computer Networking Conference, Jun 2021, Ibiza (virtual), Spain. pp.1-8, ⟨10.1109/MedComNet52149.2021.9501272⟩. ⟨hal-03376168⟩
75 Consultations
75 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More