MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA - Archive ouverte HAL
Article Dans Une Revue Biometrika Année : 2022

MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA

Résumé

Variable importance measures are the main tools to analyze the black-box mechanisms of random forests. Although the mean decrease accuracy (MDA) is widely accepted as the most efficient variable importance measure for random forests, little is known about its statistical properties. In fact, the definition of MDA varies across the main random forest software. In this article, our objective is to rigorously analyze the behavior of the main MDA implementations. Consequently, we mathematically formalize the various implemented MDA algorithms, and then establish their limits when the sample size increases. This asymptotic analysis reveals that these MDA versions differ as importance measures, since they converge towards different quantities. More importantly, we break down these limits into three components: the first two terms are related to Sobol indices, which are well-defined measures of a covariate contribution to the response variance, widely used in the sensitivity analysis field, as opposed to the third term, whose value increases with dependence within covariates. Thus, we theoretically demonstrate that the MDA does not target the right quantity to detect influential covariates in a dependent setting, a fact that has already been noticed experimentally. To address this issue, we define a new importance measure for random forests, the Sobol-MDA, which fixes the flaws of the original MDA, and consistently estimates the accuracy decrease of the forest retrained without a given covariate, but with an efficient computational cost. The Sobol-MDA empirically outperforms its competitors on both simulated and real data for variable selection. An open source implementation in R and C++ is available online.
Fichier principal
Vignette du fichier
arxiv_sobolMDA.pdf (1.08 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03151611 , version 1 (25-02-2021)
hal-03151611 , version 2 (16-11-2021)
hal-03151611 , version 3 (28-02-2022)

Identifiants

Citer

Clément Bénard, Sébastien da Veiga, Erwan Scornet. MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA. Biometrika, 2022, 109, pp.881-900. ⟨hal-03151611v3⟩
371 Consultations
701 Téléchargements

Altmetric

Partager

More