Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems

Résumé

We study the convergence of the ADMM (Alternating Direction Method of Multipliers) algorithm on a broad range of penalized regression problems including the Lasso, Group-Lasso and Graph-Lasso,(isotropic) TV-L1, Sparse Variation, and others. First, we establish a fixed-point iteration –via a nonlinear operator– which is equivalent to the ADMM iterates. We then show that this nonlinear operator is Fréchet-differentiable almost everywhere and that around each fixed point, Q-linear convergence is guaranteed, provided the spectral radius of the Jacobian of the operator at the fixed point is less than 1 (a classical result on stability). Moreover, this spectral radius is then a rate of convergence for the ADMM algorithm. Also, we show that the support of the split variable can be identified after finitely many iterations. In the anisotropic cases, we show that for sufficiently large values of the tuning parameter, we recover the optimal rates in terms of Friedrichs angles, that have appeared recently in the literature. Empirical results on various problems are also presented and discussed.
Fichier principal
Vignette du fichier
paper.pdf (404.73 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01265372 , version 1 (01-02-2016)

Identifiants

  • HAL Id : hal-01265372 , version 1

Citer

Elvis Dohmatob, Michael Eickenberg, Bertrand Thirion, Gaël Varoquaux. Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems. ICASSP, International Conference on Acoustics, Speach, and Signal Processing, Mar 2016, http://icassp2016.org/default.asp, China. ⟨hal-01265372⟩
315 Consultations
405 Téléchargements

Partager

Gmail Facebook X LinkedIn More