Universality laws for Gaussian mixtures in generalized linear models - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Universality laws for Gaussian mixtures in generalized linear models

Résumé

Let (x i , y i) i=1,...,n denote independent samples from a general mixture distribution c∈C ρ c P x c , and consider the hypothesis class of generalized linear models ŷ = F (Θ x). In this work, we investigate the asymptotic joint statistics of the family of generalized linear estimators (Θ 1 ,. .. , Θ M) obtained either from (a) minimizing an empirical risk Rn (Θ; X, y) or (b) sampling from the associated Gibbs measure exp(−βn Rn (Θ; X, y)). Our main contribution is to characterize under which conditions the asymptotic joint statistics of this family depends (on a weak sense) only on the means and covariances of the class conditional features distribution P x c. In particular, this allow us to prove the universality of different quantities of interest, such as the training and generalization errors, redeeming a recent line of work in high-dimensional statistics working under the Gaussian mixture hypothesis. Finally, we discuss the applications of our results to different machine learning tasks of interest, such as ensembling and uncertainty quantification.
Fichier principal
Vignette du fichier
2302.08933.pdf (985.63 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04019692 , version 1 (08-03-2023)

Identifiants

Citer

Yatin Dandi, Ludovic Stephan, Florent Krzakala, Bruno Loureiro, Lenka Zdeborová. Universality laws for Gaussian mixtures in generalized linear models. 2023. ⟨hal-04019692⟩
15 Consultations
18 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More