Efficient online learning with kernels for adversarial large scale problems - Archive ouverte HAL Access content directly
Preprints, Working Papers, ... Year : 2019

Efficient online learning with kernels for adversarial large scale problems

Abstract

We are interested in a framework of online learning with kernels for low-dimensional but large-scale and potentially adversarial datasets. We study the computational and theoretical performance of online variations of kernel Ridge regression. Despite its simplicity, the algorithm we study is the first to achieve the optimal regret for a wide range of kernels with a per-round complexity of order $n^\alpha$ with $\alpha < 2$. The algorithm we consider is based on approximating the kernel with the linear span of basis functions. Our contributions is two-fold: 1) For the Gaussian kernel, we propose to build the basis beforehand (independently of the data) through Taylor expansion. For $d$-dimensional inputs, we provide a (close to) optimal regret of order $O((\log n)^{d+1})$ with per-round time complexity and space complexity $O((\log n)^{2d})$. This makes the algorithm a suitable choice as soon as $n \gg e^d$ which is likely to happen in a scenario with small dimensional and large-scale dataset; 2) For general kernels with low effective dimension, the basis functions are updated sequentially in a data-adaptive fashion by sampling Nyström points. In this case, our algorithm improves the computational trade-off known for online kernel regression.
Fichier principal
Vignette du fichier
Efficient_online_learning_with_Kernels_for_adversarial_large_scale_problems__preprint.pdf (819.32 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02019402 , version 1 (19-02-2019)
hal-02019402 , version 2 (28-05-2019)

Identifiers

Cite

Rémi Jézéquel, Pierre Gaillard, Alessandro Rudi. Efficient online learning with kernels for adversarial large scale problems. 2019. ⟨hal-02019402v2⟩
171 View
184 Download

Altmetric

Share

Gmail Facebook X LinkedIn More