Pré-Publication, Document De Travail Année : 2025

When majority rules, minority loses: bias amplification of gradient descent

Résumé

Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minority learning tasks, showing how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features. Assuming population and variance imbalance, our analysis reveals three key findings: (i) the close proximity between ``full-data'' and stereotypical predictors, (ii) the dominance of a region where training the entire model tends to merely learn the majority traits, and (iii) a lower bound on the additional training required. Our results are illustrated through experiments in deep learning for tabular and image classification tasks.

Fichier principal
Vignette du fichier
bias_ampli/bias_amplification.pdf (1.16 Mo) Télécharger le fichier
bias_ampli/bias_amplification.toc (3.86 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05072199 , version 1 (18-05-2025)
hal-05072199 , version 2 (17-10-2025)

Licence

Identifiants

  • HAL Id : hal-05072199 , version 1

Citer

François Bachoc, Jérôme Bolte, Ryan Boustany, Jean-Michel Loubes. When majority rules, minority loses: bias amplification of gradient descent. 2025. ⟨hal-05072199v1⟩
337 Consultations
175 Téléchargements

Partager

  • More