Action Masking for Safer Model-Free Building Energy Management - Archive ouverte HAL Accéder directement au contenu
Poster De Conférence Année : 2023

Action Masking for Safer Model-Free Building Energy Management

Résumé

ACTION MASKING TO ENFORCE RULES ON THE AGENT * Cannot charge (discharge) a full (empty) battery * Cooling system switched off from 10 PM to 5 AM * Cooling system must stay ON if T indoor > 26.5 o C The agent is trained using PPO, a popular DRL algorithm The action mask constrains the exploration space by dynamically limiting the actions the agent can take. MASKED AGENTS CAN OUTPERFORM DIRECT RL AGENTS Key Results 1. Both DRL controllers achieved a lower cost compared to the baseline RBC. 2. The direct RL controller led to a significantly worse comfort score. 3. Action masking achieved a similar comfort score to the baseline while reducing costs. Conclusions 1. The Direct RL controller prioritized a lower energy bill over thermal comfort (local optima) due to the lack of constraints. 2. The use of Action Masking resulted in a policy that reduced the energy bill while respecting thermal comfort rules, without any modifications to the reward function or hyperparameters.
Fichier principal
Vignette du fichier
Poster_RLEM_23.pdf (1.01 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04299564 , version 1 (22-11-2023)

Identifiants

  • HAL Id : hal-04299564 , version 1

Citer

Sharath Ram Kumar, Rémy Rigo-Mariani, Benoit Delinchant, Arvind Easwaran. Action Masking for Safer Model-Free Building Energy Management. ACM SIGEnergy Workshop on Reinforcement Learning for Energy Management in Buildings & Cities (RLEM), Nov 2023, Istanbul, Turkey. ⟨hal-04299564⟩
47 Consultations
21 Téléchargements

Partager

Gmail Facebook X LinkedIn More