We consider numerical methods for thermodynamic sampling, i.e. computing sequences of points which are distributed according to the Gibbs-Boltzmann distribution, using Langevin dynamics and overdamped Langevin dynamics (Brownian dynamics). A wide variety of numerical methods for Langevin dynamics may be constructed based on splitting the stochastic differential equations into various component parts, each of which may be propagated exactly. Each such method may be viewed as generating samples according to an associated invariant measure that differs from the exact canonical invariant measure by a stepsize-dependent perturbation. We provide error estimates a la Talay-Tubaro on the invariant distribution for small stepsize, and compare the sampling bias obtained for various choices of splitting method. We further investigate the overdamped limit and apply the methods in the context of driven systems where the goal is sampling with respect to a non-equilibrium steady state. Our analyses are illustrated by numerical experiments.