Efficient online learning with kernels for adversarial large scale problems - Archive ouverte HAL
Pré-Publication, Document De Travail Année : 2019

Efficient online learning with kernels for adversarial large scale problems

Résumé

We are interested in a framework of online learning with kernels for low-dimensional but large-scale and potentially adversarial datasets. We study the computational and theoretical performance of online variations of kernel Ridge regression. Despite its simplicity, the algorithm we study is the first to achieve the optimal regret for a wide range of kernels with a per-round complexity of order $n^\alpha$ with $\alpha < 2$. The algorithm we consider is based on approximating the kernel with the linear span of basis functions. Our contributions is two-fold: 1) For the Gaussian kernel, we propose to build the basis beforehand (independently of the data) through Taylor expansion. For $d$-dimensional inputs, we provide a (close to) optimal regret of order $O((\log n)^{d+1})$ with per-round time complexity and space complexity $O((\log n)^{2d})$. This makes the algorithm a suitable choice as soon as $n \gg e^d$ which is likely to happen in a scenario with small dimensional and large-scale dataset; 2) For general kernels with low effective dimension, the basis functions are updated sequentially in a data-adaptive fashion by sampling Nyström points. In this case, our algorithm improves the computational trade-off known for online kernel regression.
Fichier principal
Vignette du fichier
Efficient_online_learning_with_Kernels_for_adversarial_large_scale_problems__preprint.pdf (819.32 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02019402 , version 1 (19-02-2019)
hal-02019402 , version 2 (28-05-2019)

Identifiants

Citer

Rémi Jézéquel, Pierre Gaillard, Alessandro Rudi. Efficient online learning with kernels for adversarial large scale problems. 2019. ⟨hal-02019402v2⟩
192 Consultations
201 Téléchargements

Altmetric

Partager

More