Optimal survey schemes for stochastic gradient descent with applications to M-estimation - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue ESAIM: Probability and Statistics Année : 2019

Optimal survey schemes for stochastic gradient descent with applications to M-estimation

Résumé

Iterative stochastic approximation methods are widely used to solve M-estimation problems, in the context of predictive learning in particular. In certain situations that shall be undoubtedly more and more common in the Big Data era, the datasets available are so massive that computing statistics over the full sample is hardly feasible, if not unfeasible. A natural and popular approach to gradient descent in this context consists in substituting the “full data” statistics with their counterparts based on subsamples picked at random of manageable size. It is the main purpose of this paper to investigate the impact of survey sampling with unequal inclusion probabilities on stochastic gradient descent-based M-estimation methods. Precisely, we prove that, in presence of some a priori information, one may significantly increase statistical accuracy in terms of limit variance, when choosing appropriate first order inclusion probabilities. These results are described by asymptotic theorems and are also supported by illustrative numerical experiments.
Fichier principal
Vignette du fichier
ps180001.pdf (1.04 Mo) Télécharger le fichier
Origine : Publication financée par une institution
Loading...

Dates et versions

hal-02078108 , version 1 (29-08-2020)

Identifiants

Citer

Stéphan Clémençon, Patrice Bertail, Emilie Chautru, Guillaume Papa. Optimal survey schemes for stochastic gradient descent with applications to M-estimation. ESAIM: Probability and Statistics, 2019, 23, pp.310-337. ⟨10.1051/ps/2018021⟩. ⟨hal-02078108⟩
147 Consultations
121 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More