The cost of learning fast with reinforcement learning for edge cache allocation - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

The cost of learning fast with reinforcement learning for edge cache allocation

Résumé

We study data-driven cache allocation in MultiTenant Edge Computing: a Network Operator (NO) owns storage at the Edge and dynamically allocates it to third party application Content Providers (CPs). CPs can cache a part of their catalog and satisfy locally users’ requests, thus reducing inter-domain traffic. The objective of the NO is to find the optimal cache allocation, which minimizes the total interdomain traffic bandwidth, which constitutes an operational cost. Since CPs’ traffic is encrypted, NO’s allocation strategy is based solely on the amount of traffic measured. In this exploratory work, we solve this problem via ReinforcementLearning (RL). RL has mainly been intended to be trained in simulation, before applying it in real scenarios. We instead employ RL online, training it directly while the system is hot and running. An important factor emerges in this case: in order to learn the optimal cache allocation, the NO needs to perturb the allocation several times and measure how the inter-domain traffic changes; when perturbing the allocation, the NO has to pay a perturbation cost. While it has no physical meaning in simulation, it cannot be ignored in a hot and running system. We explore in this work the trade-off between perturbing a lot the system in order to learn a good allocation faster, or learning slower to reduce the perturbation cost. We show results from simulation and make the entire code available as open-source.
Fichier principal
Vignette du fichier
1570654680-1.pdf (266.71 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02542133 , version 1 (14-04-2020)
hal-02542133 , version 2 (30-08-2020)

Identifiants

  • HAL Id : hal-02542133 , version 2

Citer

Theo Bouganim, Andrea Araldo, Antoine Lavignotte, Nessim Oussedik, Gabriel Guez. The cost of learning fast with reinforcement learning for edge cache allocation. ITC 2020: 32nd International Teletraffic Congress, PhD Workshop, Sep 2020, Osaka, Japan. ⟨hal-02542133v2⟩
188 Consultations
96 Téléchargements

Partager

Gmail Facebook X LinkedIn More