Highly adaptive linear actor-critic for lightweight energy-harvesting IoT applications - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Low Power Electronics and Applications Année : 2021

Highly adaptive linear actor-critic for lightweight energy-harvesting IoT applications

Résumé

Reinforcement learning (RL) has received much attention in recent years due to its adaptability to unpredictable events such as harvested energy and workload, especially in the context of edge computing for Internet-of-Things (IoT) nodes. Due to limited resources in IoT nodes, it is difficult to achieve self-adaptability. This paper studies online reactivity issues of fixed learning rate in the linear actor-critic (LAC) algorithm for transmission duty-cycle control. We propose the LAC-AB algorithm that introduces into the LAC algorithm an adaptive learning rate called Adam for actor update to achieve better adaptability. We introduce a definition of “convergence” when quantitative analysis of convergence is performed. Simulation results using real-life one-year solar irradiance data indicate that, unlike the conventional setups of two decay rate $\beta_1$,$\beta_2$ of Adam, smaller $\beta_1$ such as 0.2–0.4 are suitable for power-failure-sensitive applications and 0.5–0.7 for latency-sensitive applications with $\beta_2$ $\in$ [0.1,0.3]. LAC-AB improves the time of reactivity by 68.5–88.1% in our application; it also fine-tunes the initial learning rate for the initial state and improves the time of fine-tuning by 78.2–84.3%, compared to the LAC. Besides, the number of power failures is drastically reduced to zero or a few occurrences over 300 simulations.
Fichier principal
Vignette du fichier
jlpea-1142703-english.pdf (3.02 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

cea-03534312 , version 1 (19-01-2022)

Licence

Paternité

Identifiants

Citer

Jean-Frédéric Christmann, Sota Sawaguchi, Suzanne Lesecq. Highly adaptive linear actor-critic for lightweight energy-harvesting IoT applications. Journal of Low Power Electronics and Applications, 2021, 11 (2), pp.17. ⟨10.3390/jlpea11020017⟩. ⟨cea-03534312⟩
44 Consultations
48 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More