CMA-ES is one of the state of the art evolutionary optimization methods because of its capacity to adapt covariance to information geometry. It uses prior information to form a best guess about the distribution of the minimum. We show this can be reformulated as a Bayesian optimization problem for the sampling of the optimum. Thanks to Normal Inverse Wishart (NIW) distribution, that is a conjugate prior for the multi variate normal distribution, we can derive a numerically efficient algorithm Bayesian CMA-ES that obtains similar performance as the traditional CMA-ES on multiple benchmarks and provides a new justification for the CMA-ES updates equations. This novel paradigm for Bayesian CMA-ES provides a powerful bridge between evolutionary and Bayesian optimization, showing the profound similarities and connections between these traditionally opposed methods and opening horizon for variations and mix strategies on these methods.