This paper develops a backtesting procedure that evaluates how well ESG ratings help in predicting a company's idiosyncratic risk. Technically, the inference is based on extending the conditional predictive ability test of Giacomini and White (2006) to a panel data setting. We apply our methodology to the forecasting of stock returns idiosyncratic volatility and compare two ESG rating systems from Sustainalytics and Asset4 across three investment universes (Europe, North America, and the Asia-Pacific region). The results show that the null hypothesis of no informational content in ESG ratings is strongly rejected for firms located in Europe, whereas results appear mixed in the other regions. In most configurations, we find a negative relationship between ESG ratings and idiosyncratic risk, with higher ratings predicting lower levels of idiosyncratic volatility. Furthermore, the predictive accuracy gains are generally higher when assessing the environmental dimension of the ratings. Importantly, applying the test only to firms over which there is a high degree of consensus between the ESG rating agencies leads to higher predictive accuracy gains for all three universes. Beyond providing insights into the accuracy of each of the ESG rating systems, this last result suggests that information gathered from several ESG rating providers should be cross-checked before ESG is integrated into investment processes.