On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising

Résumé

Basis Pursuit (BP), Basis Pursuit DeNoising (BPDN) and LASSO are popular procedures for identifying important predictors in the high-dimensional linear regression model Y = Xβ + ε. When ε = 0, BP can recover the sign of β when this vector is identifiable with respect to the L1 norm, while LASSO requires a much stronger irrepresentability condition. It is known that model selection properties of LASSO can be improved by hard-thresholding its estimates. In this article we support these findings by proving that thresholded LASSO, thresholded BPDN and thresholded BP can recover the sign of β if and only if β is identifiable with respect to the L1 norm. In particular when X has iid Gaussian entries and the numbers of predictors grow linearly with the sample size then these thresholded estimators can recover the sign of β if its limiting signal sparsity is below the Donoho-Tanner transition curve. This is in contrast with vanilla LASSO which, asymptotically, can recover the sign of β only if the signal sparsity tends to 0. Numerical experiments illustrate that contrary to irrepresentability condition, the identifiability condition seems to be not affected by the structure of correlations in the X matrix.
Fichier principal
Vignette du fichier
thresholded_LASSO.pdf (533.89 Ko) Télécharger le fichier
supplementary_material.pdf (252.15 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01956603 , version 1 (16-12-2018)
hal-01956603 , version 2 (30-03-2019)
hal-01956603 , version 3 (22-04-2019)
hal-01956603 , version 4 (26-06-2019)
hal-01956603 , version 5 (08-06-2020)
hal-01956603 , version 6 (09-05-2021)
hal-01956603 , version 7 (31-08-2021)

Identifiants

  • HAL Id : hal-01956603 , version 5

Citer

Patrick J C Tardivel, Malgorzata Bogdan. On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising. 2020. ⟨hal-01956603v5⟩
317 Consultations
841 Téléchargements

Partager

Gmail Facebook X LinkedIn More