Applying quantitative perturbation theory for linear operators, we prove non-asymptotic limit theorems for Markov chains whose transition kernel has a spectral gap in an arbitrary Banach algebra of functions X . The main results are concentration inequalities and Berry-Esseen bounds, obtained assuming neither reversibility nor “warm start” hypothesis: the law of the first term of the chain can be arbitrary. The spectral gap hypothesis is basically a uniform X-ergodicity hypothesis, and when X consist in regular functions this is weaker than uniform ergodicity. We show on a few examples how the flexibility in the choice of function space can be used. The constants are completely explicit and reasonable enough to make the results usable in practice, notably in MCMC methods. v2: Introduction rewritten, Section 3 applying the main results to examples improved (uniformly ergodic chains and Bernoulli convolutions have been notably added) . Main results and their proofs are unchanged.