Manifold based local classifiers: linear and nonlinear approaches - Archive ouverte HAL Access content directly
Journal Articles Journal of Signal Processing Systems Year : 2010

Manifold based local classifiers: linear and nonlinear approaches


In case of insufficient data samples in highdimensional classification problems, sparse scatters of samples tend to have many 'holes'--regions that have few or no nearby training samples from the class. When such regions lie close to inter-class boundaries, the nearest neighbors of a query may lie in the wrong class, thus leading to errors in the Nearest Neighbor classification rule. The K-local hyperplane distance nearest neighbor (HKNN) algorithm tackles this problem by approximating each class with a smooth nonlinear manifold, which is considered to be locally linear. The method takes advantage of the local linearity assumption by using the distances from a query sample to the affine hulls of query's nearest neighbors for decision making. However, HKNN is limited to using the Euclidean distance metric, which is a significant limitation in practice. In this paper we reformulate HKNN in terms of subspaces, and propose a variant, the Local Discriminative Common Vector (LDCV) method, that is more suitable for classification tasks where the classes have similar intra-class variations. We then extend both methods to the nonlinear case by mapping the nearest neighbors into a higherdimensional space where the linear manifolds are constructed. This procedure allows us to use a wide variety of distance functions in the process, while computing distances between the query sample and the nonlinear manifolds remains straightforward owing to the linear nature of the manifolds in the mapped space. We tested the proposed methods on several classification tasks, obtaining better results than both the Support Vector Machines (SVMs) and their local counterpart SVM-KNN on the USPS and Image segmentation databases, and outperforming the local SVMKNN on the Caltech visual recognition database.
Fichier principal
Vignette du fichier
Cevikalp-jsps10.pdf (440.67 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-00565007 , version 1 (10-02-2011)



Hakan Cevikalp, Diane Larlus, Mike Neamtu, William Triggs, Frédéric Jurie. Manifold based local classifiers: linear and nonlinear approaches. Journal of Signal Processing Systems, 2010, 61 (1), pp.61-73. ⟨10.1007/s11265-008-0313-4⟩. ⟨hal-00565007⟩
508 View
386 Download



Gmail Facebook Twitter LinkedIn More