Numerical accuracy of floating point computation is a well studied topic, but which has not made its way to the end-user in scientific computing. With the recent requirements for code modernization to exploit new highly parallel hardware and perform higher resolution computation, this has become one of the critical issues to address. To democratize numerical accuracy analysis, it is important to propose tools and methodologies to study large use cases in a reliable and automatic way. In this paper, we propose verificarlo, an extension to the LLVM compiler to automatically use Monte Carlo Arithmetic in a transparent way for the end-user. It supports all the major languages including C, C++ and Fortran. We also illustrate the fact that unlike source-to-source approaches, our implementation captures the influence of compiler optimizations on the numerical accuracy. Finally, we illustrate on various use cases how Monte Carlo Arithmetic using the verificarlo tool outperforms the existing approaches and is a step toward automatic numerical analysis of large scientific applications.