Following the energy transition, Réseau de Transport d'Electricité (RTE) - the French Transport System Operator- is developing new adaptive zonal automatons. Each automaton monitors a zone of the power grid thanks to an optimization algorithm. An automaton can act on the power grid configuration in case of constraints. Each automaton can receive a target setpoint from human operators to decide the appropriate actions. Examples of a target path include (un)desirable configurations in future hours or information about other zones. To make operators’ work easier, RTE needs a decision-support assistant to recommend relevant target setpoints. Reinforcement Learning (RL) is a promising method to elaborate such target setpoints. However, RL requires a high number of training iterations which is time-consuming. In this poster, we present our research to design an emulator of an automaton with a short response time, thanks to a RL approach, to afford a sufficient number of iterations and reasonable computation times during training.