Generalization Bounds in the Presence of Outliers: a Median-of-Means Study - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Generalization Bounds in the Presence of Outliers: a Median-of-Means Study

Résumé

In contrast to the empirical mean, the Median-ofMeans (MoM) is an estimator of the mean θ of a square integrable r.v. Z, around which accuratenonasymptotic confidence bounds can be built, even when Z does not exhibit a sub-Gaussian tail behavior. Thanks to the high confidence it achieves on heavy-tailed data, MoM has found various applications in machine learning, where it is used to design training procedures that are not sensitive to atypical observations. More recently, a new line of work is now trying to characterize and leverage MoM’s ability to deal with corrupted data. In this context, the present work proposes a general study of MoM’s concentration properties under the contamination regime, that provides a clear understanding of the impact of the outlier proportion and the number of blocks chosen. The analysis is extended to (multisample) U-statistics, i.e. averages over tuples of observations, that raise additional challenges due to the dependence induced. Finally, we show that the latter bounds can be used in a straightforward fashion to derive generalization guarantees for pairwise learning in a contaminated setting, and propose an algorithm to compute provably reliable decision functions.

Dates et versions

hal-03423979 , version 1 (10-11-2021)

Identifiants

Citer

Pierre Laforgue, Guillaume Staerman, Stéphan Clémençon. Generalization Bounds in the Presence of Outliers: a Median-of-Means Study. 38 th International Conference on Machine Learning (2021), Jul 2021, Virtual conference, France. ⟨hal-03423979⟩
30 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More