Behavior of linear L2-boosting algorithms in the vanishing learning rate asymptotic - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Behavior of linear L2-boosting algorithms in the vanishing learning rate asymptotic

Résumé

We investigate the asymptotic behaviour of gradient boosting algorithms when the learning rate converges to zero and the number of iterations is rescaled accordingly. We mostly consider $L^2$-boosting for regression with linear base learner as studied in \cite{BY03} and analyze also a stochastic version of the model where subsampling is used at each step \citep{F02}. We prove a deterministic limit in the vanishing learning rate asymptotic and characterize the limit as the unique solution of a linear differential equation in an infinite dimensional function space. Besides, the training and test error of the limiting procedure are thoroughly analyzed. We finally illustrate and discuss our result with a simple numerical experiment where linear $L^2$-boosting predictor is interpreted as a smoothed projection where time is related to the degrees of freedom of the linear boosting operator.
Fichier principal
Vignette du fichier
L2_boosting.pdf (941.61 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03078992 , version 1 (20-12-2020)

Identifiants

Citer

Clément Dombry, Youssef Esstafa. Behavior of linear L2-boosting algorithms in the vanishing learning rate asymptotic. 2020. ⟨hal-03078992⟩
92 Consultations
80 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More