High-dimensional analysis of double descent for linear regression with random projections - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

High-dimensional analysis of double descent for linear regression with random projections

Francis Bach
  • Fonction : Auteur
  • PersonId : 863086

Résumé

We consider linear regression problems with a varying number of random projections, where we provably exhibit a double descent curve for a fixed prediction problem, with a high-dimensional analysis based on random matrix theory. We first consider the ridge regression estimator and review earlier results using classical notions from non-parametric statistics, namely degrees of freedom, also known as effective dimensionality. We then compute asymptotic equivalents of the generalization performance (in terms of squared bias and variance) of the minimum norm least-squares fit with random projections, providing simple expressions for the double descent phenomenon.
Fichier principal
Vignette du fichier
double_descent_linear_reg_random_proj_v2.pdf (493.5 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04008311 , version 1 (01-03-2023)
hal-04008311 , version 2 (13-03-2023)

Identifiants

Citer

Francis Bach. High-dimensional analysis of double descent for linear regression with random projections. 2023. ⟨hal-04008311v2⟩
43 Consultations
259 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More