Total Variation image denoising, generally formulated in a variational setting, can be seen as a Maximum A Posteriori (MAP) Bayesian estimate relying on a simple explicit image prior. In this formulation, the denoised image is the most likely image of the posterior distribution, which favors regularity and produces staircasing artifacts: in regions where smooth-varying intensities would be expected, constant zones appear separated by artificial boundaries. In this paper, we propose to use the Least Square Error (LSE) criterion instead of the MAP. This leads to a new denoising method called TV-LSE, that produces more realistic images by computing the expectation of the posterior distribution. We describe a Monte-Carlo Markov Chain algorithm based on Metropolis scheme, and provide an efficient convergence criterion. We also discuss the properties of TV-LSE, and show in particular that it does not suffer from the staircasing effect.