RKHS Weightings of Functions - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

RKHS Weightings of Functions

Gabriel Dubé
Mario Marchand
  • Fonction : Auteur
  • PersonId : 1291330

Résumé

We examine a new general machine learning model for binary classification by considering the expected weighted output of a parameterized predictor, where the weighting belongs to an RKHS of functions over the parameters. This weighting can be learned using various algorithms. One of them is Stochastic Functional Gradient Descent (SFGD), which itera- tively samples a parameter and some training data, and calculates an approximation of the functional gradient of the loss. Using the stability properties of the algorithm, we show that convergence is guaranteed, under mild assumptions, with rate O(1/√m) on the number of examples needed for learning. Further theoretical analysis, based on the Rademacher com- plexity of the proposed class of predictors, provides a similar bound on the generalization error. We also present three alternate learning algorithms, and a procedure for pruning the model using the Lasso. We prove an error bound for the resulting sparse predictor. Finally, we run experiments using simple instantiations of the model to showcase its usability, and compare the learning algorithms among one another, and to the state of the art.
Fichier principal
Vignette du fichier
JMLR2023 submission.pdf (790.13 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04236058 , version 1 (10-10-2023)
hal-04236058 , version 2 (23-07-2024)

Identifiants

  • HAL Id : hal-04236058 , version 2

Citer

Gabriel Dubé, Mario Marchand. RKHS Weightings of Functions. 2024. ⟨hal-04236058v2⟩
48 Consultations
70 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More