Pré-Publication, Document De Travail Année : 2025

Federated Majorize-Minimization: Beyond Parameter Aggregation

Résumé

This paper proposes a unified approach for designing stochastic optimization algorithms that robustly scale to the federated learning setting. Our work studies a class of Majorize-Minimization (MM) problems, which possesses a linearly parameterized family of majorizing surrogate functions. This framework encompasses (proximal) gradient-based algorithms for (regularized) smooth objectives, the Expectation Maximization algorithm, and many problems seen as variational surrogate MM. We show that our framework motivates a unifying algorithm called Stochastic Approximation Stochastic Surrogate MM (SA-SSMM), which includes previous stochastic MM procedures as special instances. We then extend SA-SSMM to the federated setting, while taking into consideration common bottlenecks such as data heterogeneity, partial participation, and communication constraints; this yields FedMM. The originality of FedMM is to learn locally and then aggregate information characterizing the surrogate majorizing function, contrary to classical algorithms which learn and aggregate the original parameter. Finally, to showcase the flexibility of this methodology beyond our theoretical setting, we use it to design an algorithm for computing optimal transport maps in the federated setting.

Fichier principal
Vignette du fichier
hal_FedMM__arxiv_submission.pdf (1.8 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05189632 , version 1 (28-07-2025)

Licence

Identifiants

Citer

Aymeric Dieuleveut, Gersende Fort, Mahmoud Hegazy, Hoi-To Wai. Federated Majorize-Minimization: Beyond Parameter Aggregation. 2025. ⟨hal-05189632⟩
1259 Consultations
269 Téléchargements

Altmetric

Partager

  • More