On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising
Résumé
Basis Pursuit (BP), Basis Pursuit DeNoising (BPDN) and LASSO are popular procedures for identifying
important predictors in the high-dimensional linear regression model Y = Xβ + ε. When ε = 0, BP can
recover the sign of β when this vector is identifiable with respect to the L1 norm, while LASSO requires
a much stronger irrepresentability condition. It is known that model selection properties of LASSO can
be improved by hard-thresholding its estimates. In this article we support these findings by proving that
thresholded LASSO, thresholded BPDN and thresholded BP can recover the sign of β if and only if β is
identifiable with respect to the L1 norm. In particular when X has iid Gaussian entries and the numbers
of predictors grow linearly with the sample size then these thresholded estimators can recover the sign of β
if its limiting signal sparsity is below the Donoho-Tanner transition curve. This is in contrast with vanilla
LASSO which, asymptotically, can recover the sign of β only if the signal sparsity tends to 0. Numerical
experiments illustrate that contrary to irrepresentability condition, the identifiability condition seems to be
not affected by the structure of correlations in the X matrix.
Fichier principal
thresholded_LASSO.pdf (533.89 Ko)
Télécharger le fichier
supplementary_material.pdf (252.15 Ko)
Télécharger le fichier
Origine | Fichiers produits par l'(les) auteur(s) |
---|
Loading...