On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year : 2019

On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising

Malgorzata Bogdan
  • Function : Author

Abstract

In the high-dimensional regression model Y = Xβ + ε, we provide new theoretical results on the probability of recovering the sign of β by the Least Absolute Selection and Shrinkage Operator (LASSO) and by the thresholded LASSO. It is well known that "irrepresentability" is a necessary condition for LASSO to recover the sign of β with a large probability. In this article we extend this result by providing a tight upper bound for the probability of LASSO sign recovery. This upper bound is smaller than 1/2 when the irrepresentable condition does not hold and thus generalizes Theorem 2 of Wainwright [27]. The bound depends on the tuning parameter λ and is attained when non-null components of β tend to infinity; its value is equal to the limit of the probability that every null component of β is correctly estimated at 0. Consequently, this bound makes it possible to select λ so as to control the probability of at least one false discovery. The "irrepresentability" is a stringent necessary condition to recover the sign of β by LASSO which can be substantially relaxed when LASSO estimates are additionally filtered out with an appropriately selected threshold. In this article we provide new theoretical results on thresholded LASSO and thresholded Basis Pursuit DeNoising (BPDN) in the asymptotic setup under which X is fixed and non-null components of β tend to infinity. Compared to the classical asymptotics, where X is a n × p matrix and both n and p tend to +∞, our approach allows for reduction of the technical burden. Our main Theorem takes a simple form: When non-null components of β are sufficiently large, appropriately thresholded LASSO or thresholded BPDN can recover the sign of β if and only if β is identifiable with respect to the L1 norm, i.e. If Xγ = Xβ and γ = β then γ 1 > β 1. To illustrate our results we present examples of irrepresentability and identifiability curves for some selected design matrices X. These curves provide the proportion of k sparse vectors β for which the irrep-* Corresponding author: tardivel@math.uni.wroc.pl 1 resentability and identifiability conditions hold. Our examples illustrate that "irrepresentability" is a much stronger condition than "identifiability", especially when the entries in each row of X are strongly correlated. Finally, we illustrate how the knockoff methodology [1, 8] can be used to select an appropriate threshold and that thresholded BPDN and LASSO can recover the sign of β with a larger probability than adaptive LASSO [32].
Fichier principal
Vignette du fichier
thresholded_LASSO_HAL_V4.pdf (555.85 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01956603 , version 1 (16-12-2018)
hal-01956603 , version 2 (30-03-2019)
hal-01956603 , version 3 (22-04-2019)
hal-01956603 , version 4 (26-06-2019)
hal-01956603 , version 5 (08-06-2020)
hal-01956603 , version 6 (09-05-2021)
hal-01956603 , version 7 (31-08-2021)

Identifiers

  • HAL Id : hal-01956603 , version 4

Cite

Patrick J C Tardivel, Malgorzata Bogdan. On the sign recovery by LASSO, thresholded LASSO and thresholded Basis Pursuit Denoising. 2019. ⟨hal-01956603v4⟩
312 View
822 Download

Share

Gmail Facebook X LinkedIn More