Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation

Résumé

In this manuscript we consider the problem of generalized linear estimation on Gaussian mixture data with labels given by a single-index model. Our first result is a sharp asymptotic expression for the test and training errors in the high-dimensional regime. Motivated by the recent stream of results on the Gaussian universality of the test and training errors in generalized linear estimation, we ask ourselves the question: "when is a single Gaussian enough to characterize the error?". Our formula allow us to give sharp answers to this question, both in the positive and negative directions. More precisely, we show that the sufficient conditions for Gaussian universality (or lack of thereof) crucially depend on the alignment between the target weights and the means and covariances of the mixture clusters, which we precisely quantify. In the particular case of least-squares interpolation, we prove a strong universality property of the training error, and show it follows a simple, closed-form expression. Finally, we apply our results to real datasets, clarifying some recent discussion in the literature about Gaussian universality of the errors in this context.
Fichier principal
Vignette du fichier
2302.08923.pdf (3.32 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04019705 , version 1 (08-03-2023)

Identifiants

Citer

Luca Pesce, Florent Krzakala, Bruno Loureiro, Ludovic Stephan. Are Gaussian data all you need? Extents and limits of universality in high-dimensional generalized linear estimation. 2023. ⟨hal-04019705⟩
4 Consultations
12 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More