Online kernel principal component analysis: a reduced-order model
Résumé
Kernel principal component analysis (kernel-PCA) is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques, the principal component analysis. In this paper, we propose an online algorithm for kernel-PCA. To this end, we examine a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe. As with most kernel-based machines, the model order equals the number of available observations. To provide an online scheme, we propose to control the model order. We discuss theoretical results, such as an upper bound on the error of approximating the principal functions with the reduced-order model. We derive a recursive algorithm to discover the first principal axis, and extend it to multiple axes. Experimental results demonstrate the effectiveness of the proposed approach, both on synthetic data set and on images of handwritten digits, with comparison to classical kernel-PCA and iterative kernel-PCA.
Mots clés
non-stationarity
machine learning
pre-image problem
sparsity
adaptive filtering
data analysis
function approximation
Kernel
iterative kernel-PCA
classical kernel-PCA
handwritten digit image
synthetic data set
Dictionaries
Eigenvalues and eigenfunctions
Principal component analysis
reduced order systems
online kernel principal component analysis
reduced-order model
dimensionality reduction techniques
online algorithm
Oja rule
linear principal axe extraction
kernel-based machines
principal function approximation
Data models
Training data
reproducing kernel
Oja's rule
recursive algorithm.
Index Terms-Principal component analysis
recursive algorithm
Algorithm design and analysis
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|
Loading...