In this paper, we study first the problem of nonparametric estimation of the stationary density $f$ of a discrete-time Markov chain $(X_i)$. We consider a collection of projection estimators on finite dimensional linear spaces. We select an estimator among the collection by minimizing a penalized contrast. The same technique enables to estimate the density $g$ of $(X_i, X_{i+1})$ and so to provide an adaptive estimator of the transition density $\pi=g/f$. We give bounds in $L^2$ norm for these estimators and we show that they are adaptive in the minimax sense over a large class of Besov spaces. Some examples and simulations are also provided.