Reinforcement Learning for Optimal HVAC Control: From Theory to Real-World Applications
Résumé
The HVAC system accounted for a significant portion of the building's energy consumption, resulting in enormous CO 2 emissions. Among the numerous HVAC control methods, reinforcement learning (RL) gives the ability to control complex systems without requiring an explicit model of the building's thermal dynamics. This study conducted a concise review of previous research on the application of RL to HVAC systems in buildings, it offered a thorough explanation of the theoretical foundations of RL and a summary of several recent RL studies that employ a particular variant of each main component of the RL: environment, state-space, action-space, rewards function, number of time steps and training episodes. Most studies construct the training environment as a stationary MDP due to the use of a predefined single sequence of transitions for non-action-controllable state vector components (e.g., outdoor temperature and occupancy schedule). This type of MDP is solved using tabular RL and DRL algorithms. Future research should focus on using the Meta-RL approach for HVAC systems, which solves the problem of non-stationarity in the environment (non-stationarity-MDP) where the sequence of transitions for (outdoor temperature and occupancy schedule) are changeable, as is the case during the real-world implementation of the RL controller in buildings.