Landmark-based Ensemble Learning with Random Fourier Features and Gradient Boosting - Archive ouverte HAL Access content directly
Conference Papers Year : 2020

Landmark-based Ensemble Learning with Random Fourier Features and Gradient Boosting

Léo Gautheron
Amaury Habrard
Guillaume Metzler
Emilie Morvant
Marc Sebban
Valentina Zantedeschi
  • Function : Author
  • PersonId : 982876

Abstract

This paper jointly leverages two state-of-the-art learning strategies gradient boosting (GB) and kernel Random Fourier Features (RFF)-to address the problem of kernel learning. Our study builds on a recent result showing that one can learn a distribution over the RFF to produce a new kernel suited for the task at hand. For learning this distribution, we exploit a GB scheme expressed as ensembles of RFF weak learners, each of them being a kernel function designed to fit the residual. Unlike Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it from the training data as a weighted sum of RFF. This strategy allows one to build a classifier based on a small ensemble of learned kernel "landmarks" better suited for the underlying application. We conduct a thorough experimental analysis to highlight the advantages of our method compared to both boosting-based and kernel-learning state-of-the-art methods.
Fichier principal
Vignette du fichier
ECML_GBRFF.pdf (1.37 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-02900044 , version 1 (15-07-2020)

Identifiers

  • HAL Id : hal-02900044 , version 1

Cite

Léo Gautheron, Pascal Germain, Amaury Habrard, Guillaume Metzler, Emilie Morvant, et al.. Landmark-based Ensemble Learning with Random Fourier Features and Gradient Boosting. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Sep 2020, Ghent, Belgium. ⟨hal-02900044⟩
218 View
295 Download

Share

Gmail Mastodon Facebook X LinkedIn More