A large-sample theory for infinitesimal gradient boosting - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year : 2022

A large-sample theory for infinitesimal gradient boosting

Abstract

Infinitesimal gradient boosting is defined as the vanishing-learning-rate limit of the popular tree-based gradient boosting algorithm from machine learning (Dombry and Duchamps, 2021). It is characterized as the solution of a nonlinear ordinary differential equation in a infinite-dimensional function space where the infinitesimal boosting operator driving the dynamics depends on the training sample. We consider the asymptotic behavior of the model in the large sample limit and prove its convergence to a deterministic process. This infinite population limit is again characterized by a differential equation that depends on the population distribution. We explore some properties of this population limit: we prove that the dynamics makes the test error decrease and we consider its long time behavior.

Dates and versions

hal-03795853 , version 1 (04-10-2022)

Identifiers

Cite

Clement Dombry, Jean-Jil Duchamps. A large-sample theory for infinitesimal gradient boosting. 2022. ⟨hal-03795853⟩
23 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More