On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising - Archive ouverte HAL
Pré-Publication, Document De Travail Année : 2021

On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising

Résumé

Basis Pursuit (BP), Basis Pursuit DeNoising (BPDN), and LASSO are popular methods for identifying important predictors in the high-dimensional linear regression model Y = Xβ + ε. By definition, when ε = 0, BP uniquely recovers β when Xβ = Xb and β different than b implies L1 norm of β is smaller than the L1 norm of b (identifiability condition). Furthermore, LASSO can recover the sign of β only under a much stronger irrepresentability condition. Meanwhile, it is known that the model selection properties of LASSO can be improved by hard-thresholding its estimates. This article supports these findings by proving that thresholded LASSO, thresholded BPDN and thresholded BP recover the sign of β in both the noisy and noiseless cases if and only if β is identifiable and large enough. In particular, if X has iid Gaussian entries and the number of predictors grows linearly with the sample size, then these thresholded estimators can recover the sign of β when the signal sparsity is asymptotically below the Donoho-Tanner transition curve. This is in contrast to the regular LASSO, which asymptotically, recovers the sign of β only when the signal sparsity tends to 0. Numerical experiments show that the identifiability condition, unlike the irrepresentability condition, does not seem to be affected by the structure of the correlations in the X matrix.
Fichier principal
Vignette du fichier
Sign_recovery_LASSO_thresholdedd_LASSO.pdf (780.45 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01956603 , version 1 (16-12-2018)
hal-01956603 , version 2 (30-03-2019)
hal-01956603 , version 3 (22-04-2019)
hal-01956603 , version 4 (26-06-2019)
hal-01956603 , version 5 (08-06-2020)
hal-01956603 , version 6 (09-05-2021)
hal-01956603 , version 7 (31-08-2021)

Identifiants

  • HAL Id : hal-01956603 , version 6

Citer

Patrick J C Tardivel, Malgorzata Bogdan. On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising. 2021. ⟨hal-01956603v6⟩
381 Consultations
923 Téléchargements

Partager

More