This paper introduces a novel theoretically sound approach for the celebrated CMA-ES algorithm. Assuming the parameters of the multi variate normal distribution for the minimum follow a conjugate prior distribution, we can derive the optimal update at each iteration step thanks to Bayesian statistics. Update formulae are very similar to the vanilla (μ/λ) CMA-ES. We also use variance contraction and dilatation to accommodate for local and global search. As a result, this Bayesian framework provides a justification for the update of the CMA-ES algorithm and gives a new version of CMA-ES assuming normal-Inverse Wishart prior that has similar convergence speed and efficiency as the vanilla (μ/λ) CMA-ES on test functions ranging from cone, Rastrigin to Schwefel 1 and 2 functions.