We propose a stochastic control problem to control cooperatively Thermostatically Controlled Loads (TCLs) to promote power balance in electricity networks. We develop a method to solve this stochastic control problem with a decentralized architecture, in order to respect privacy of individual users and to reduce both the telecommunications and the computational burden compared to the setting of an omniscient central planner. This paradigm is called federated learning in the machine learning community, see [YFY20], therefore we refer to this problem as a federated stochastic control problem. The optimality conditions are expressed in the form of a high-dimensional Forward-Backward Stochastic Differential Equation (FBSDE), which is decomposed into smaller FBSDEs modeling the optimal behaviors of the aggregate population of TCLs of individual agents. In particular, we show that these FBSDEs fully characterize the Nash equilibrium of a stochastic Stackelberg differential game. In this game, a coordinator (the leader) aims at controlling the aggregate behavior of the population, by sending appropriate signals, and agents (the followers) respond to this signal by optimizing their storage system locally. A mean-field-type approximation is proposed to circumvent telecommunication constraints and privacy issues. Convergence results and error bounds are obtained for this approximation depending on the size of the population of TCLs. A numerical illustration is provided to show the interest of the control scheme and to exhibit the convergence of the approximation. An implementation which answers practical industrial challenges to deploy such a scheme is presented and discussed.