We prove the consistency of Galerkin methods to solve Poisson equations where the differential operator under consideration is the generator of the Langevin dynamics. We show in particular how the hypocoercive nature of this operator can be used at the discrete level to first prove the invertibility of the rigidity matrix, and next provide error bounds on the approximation of the solution of the Poisson equation. We present general convergence results in an abstract setting, as well as explicit convergence rates for a simple one-dimensional example discretized using a tensor basis. Our theoretical findings are illustrated by numerical simulations.