Consumers in smart grids are expected to engage demand-response programs by two-way communication. This makes smart grids vulnerable to cyber attacks. In this paper, we study the false pricing attacks and model the interaction between attackers and defenders using a zero-sum Markov game, where neither player has full knowledge of the game model. A multi-agent reinforcement learning method is used to solve the Markov game and find the Nash Equilibrium policies for both players. An application to a simple radial power distribution system is worked out. The results show that the proposed algorithm can help the players find mixed strategies to maximize their long-term return.