Acceleration by Randomization: Randomized First Order Algorithms for Large-scale Convex Optimization
Résumé
We discuss the possibility to accelerate solving extremely large-scale well structured convex optimization problems by replacing computationally expensive in the large scale case deterministic first order oracles with their computationally cheap stochastic counterparts and subsequent utilizing state of the art techniques of Convex Stochastic Programming. We show that when medium-accuracy solutions are sought, there are situations where this approach allows to provably outperform the best known deterministic algorithms. This includes solving matrix games and bilinear Nash Equilibrium problems, minimizing convex polynomials over simplexes, recovering signals via L1 minimization, and eigenvalue minimization.