Variance-based importance measures in the linear regression context: Review, new insights and applications - Archive ouverte HAL
Pré-Publication, Document De Travail Année : 2023

Variance-based importance measures in the linear regression context: Review, new insights and applications

Résumé

In the context of regression analysis, importance measures are effective tools to perform feature selection or to interpret a model by ranking the most influential regressors. In particular, variance-based importance measures (VIMs) are prominent in the field of statistics, but also in the most recent field of global sensitivity analysis, due to their accessible interpretation as variance shares of the explained variable. By focusing on the linear regression model, this work aims at revisiting the overview of the most well-founded methods (some of them being rather old and sometimes, misunderstood), while clarifying their respective positioning, conditions of use, intrinsic capabilities, and interpretation. Some challenges are discussed, such as the case of dependent inputs and the case of a high input dimension. A particular set of VIMs, called the Johnson indices, has shown the potential to solve both previously mentioned challenges. Their equivalence with other known VIMs, the Lindeman-Merenda-Gold (LMG) indices, is proved in the case of two inputs. The practical relevancy of such tools is highlighted through their empirical study on simulated data, as well as public datasets. An application to classification tasks (logistic regression) is also presented in the supplementary material.
Fichier principal
Vignette du fichier
VIMlinear_HAL.pdf (1.47 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04102053 , version 1 (22-05-2023)
hal-04102053 , version 2 (02-10-2023)
hal-04102053 , version 3 (05-07-2024)

Identifiants

  • HAL Id : hal-04102053 , version 1

Citer

Laura Clouvel, Bertrand Iooss, Vincent Chabridon, Marouane Il Idrissi, Frédérique Robin. Variance-based importance measures in the linear regression context: Review, new insights and applications. 2023. ⟨hal-04102053v1⟩
666 Consultations
681 Téléchargements

Partager

More