Dual Algorithms for Large-Scale Learning Problems - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2014

Dual Algorithms for Large-Scale Learning Problems

Résumé

Now Classical First-Order (FO) algorithms of convex optimization, such as Mirror Descent algorithm or Nesterov's optimal algorithm for smooth convex optimization, are well known to have optimal (theoretical) complexity estimates which do not depend on the problem dimension. However, to attain the optimality, the domain of the problem should admit a "good proximal setup". The latter essentially means that (i) the problem domain should satisfy certain geometric conditions (or be of "favorable geometry"), and (ii) the practical use of these methods is conditioned by our ability to solve efficiently an auxiliary optimization task - computing the proximal transformation -- at each iteration of the method. More often than not these two conditions are satisfied in optimization problems arising in computational learning, what explains the fact that FO methods of proximal type recently became methods of choice when solving various learning problems. Yet, they meet their limits in several important problems such as multi-task learning with large number of tasks, where the problem domain does not exhibit favorable geometry, and learning and matrix completion problems with nuclear norm constraint, when the numerical cost of solving the auxiliary problem becomes prohibitive in large-scale problems. We propose a novel approach to solving nonsmooth optimization problems arising in learning applications where Fenchel-type representation of the objective function is available. The approach is based on applying FO algorithms to the dual problem and using the accuracy certificates supplied by the method to recover the primal solution. While suboptimal in terms of accuracy guaranties, the proposed approach does not rely upon "good proximal setup" for the primal problem but assume that the problem domain admits a Linear Optimization oracle -- the ability to efficiently maximize a linear form on the domain of the primal problem.
Fichier non déposé

Dates et versions

hal-00981874 , version 1 (23-04-2014)

Identifiants

  • HAL Id : hal-00981874 , version 1

Citer

Anatoli B. Juditsky, Arkadii S. Nemirovski. Dual Algorithms for Large-Scale Learning Problems. Theory and Practice: Dealing with Big Data and other Challenges, Informs Optimizaton Society Conference, Mar 2014, Houston, United States. ⟨hal-00981874⟩
162 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More