Small and Medium Size enterprises (SMEs) are critical actors in the fabric of the economy. Their growth is often limited by the difficulty in obtaining financing. Basel II accords enforced the obligation for banks to estimate the probability of default of their obligors. Currently used models are limited by the simplicity of their architecture and the available data. State of the art machine learning models are not widely used because they are often considered as black boxes that cannot be easily explained or interpreted. We propose a methodology to combine high predictive power and powerful explainability using various Gradient Boosting Decision Trees (GBDT) implementations and Shapley additive explanation (SHAP) values as post-prediction explanation model. This method is developed and tested using a nation-wide sample of French companies, and a history of past failures extracted from commercial court decisions. The performances of GBDT models are compared with traditional credit scoring algorithms. GBDT provides the best performances over the test sample, while being fast to train and economically sound. Results obtained from SHAP values analysis are consistent with previous socio-economic studies. Providing such a level of explainability to complex models may convince regulators to accept their use in automated credit scoring.