Straight-Through meets Sparse Recovery: the Support Exploration Algorithm - Archive ouverte HAL Access content directly
Conference Papers Year : 2024

Straight-Through meets Sparse Recovery: the Support Exploration Algorithm

Abstract

The {\it straight-through estimator} (STE) is commonly used to optimize quantized neural networks, yet its contexts of effective performance are still unclear despite empirical successes. To make a step forward in this comprehension, we apply STE to a well-understood problem: {\it sparse support recovery}. We introduce the {\it Support Exploration Algorithm} (SEA), a novel algorithm promoting sparsity, and we analyze its performance in support recovery (a.k.a. model selection) problems. SEA explores more supports than the state-of-the-art, leading to superior performance in experiments, especially when the columns of $A$ are strongly coherent. The theoretical analysis considers recovery guarantees when the linear measurements matrix $A$ satisfies the {\it Restricted Isometry Property} (RIP). The sufficient conditions of recovery are comparable but more stringent than those of the state-of-the-art in sparse support recovery. Their significance lies mainly in their applicability to an instance of the STE.
Fichier principal
Vignette du fichier
Main_SEA.pdf (1.35 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03964976 , version 1 (31-01-2023)
hal-03964976 , version 2 (07-02-2024)
hal-03964976 , version 3 (24-06-2024)

Licence

Public Domain

Identifiers

Cite

Mimoun Mohamed, François Malgouyres, Valentin Emiya, Caroline Chaux. Straight-Through meets Sparse Recovery: the Support Exploration Algorithm. ICML 2024, the 41st International Conference on Machine Learning, Jul 2024, Vienna, Austria. ⟨hal-03964976v3⟩
586 View
297 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More