Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM - Archive ouverte HAL Access content directly
Journal Articles IEEE Transactions on Neural Networks and Learning Systems Year : 2013

Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM

Abstract

Feature selection in learning to rank has recently emerged as a crucial issue. Whereas several preprocessing approaches have been proposed, only a few works have been focused on integrating the feature selection into the learning process. In this work, we propose a general framework for feature selection in learning to rank using SVM with a sparse regularization term. We investigate both classical convex regularizations such as $\ell_1$ or weighted $\ell_1$ and non-convex regularization terms such as log penalty, Minimax Concave Penalty (MCP) or $\ell_p$ pseudo norm with $p<1$. Two algorithms are proposed, first an accelerated proximal approach for solving the convex problems, second a reweighted $\ell_1$ scheme to address the non-convex regularizations. We conduct intensive experiments on nine datasets from Letor 3.0 and Letor 4.0 corpora. Numerical results show that the use of non-convex regularizations we propose leads to more sparsity in the resulting models while prediction performance is preserved. The number of features is decreased by up to a factor of six compared to the $\ell_1$ regularization. In addition, the software is publicly available on the web.
Fichier principal
Vignette du fichier
IEEESVMProximal.pdf (374 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01123818 , version 2 (20-11-2013)
hal-01123818 , version 1 (05-03-2015)
hal-01123818 , version 3 (02-07-2015)

Identifiers

Cite

Léa Laporte, Rémi Flamary, Stephane Canu, Sébastien Dejean, Josiane Mothe. Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM. IEEE Transactions on Neural Networks and Learning Systems, 2013, vol. 25 (n° 6), pp.1118-1130. ⟨10.1109/TNNLS.2013.2286696⟩. ⟨hal-01123818v3⟩
823 View
670 Download

Altmetric

Share

Gmail Facebook X LinkedIn More