Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms - Archive ouverte HAL
Proceedings/Recueil Des Communications Année : 2023

Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms

Résumé

This paper focuses on Bayesian inference in a federated learning context (FL). While several distributed MCMC algorithms have been proposed, few consider the specific limitations of FL such as communication bottlenecks and statistical heterogeneity. Recently, Federated Averaging Langevin Dynamics (FALD) was introduced, which extends the Federated Averaging algorithm to Bayesian inference. We obtain a novel tight non-asymptotic upper bound on the Wasserstein distance to the global posterior for FALD. This bound highlights the effects of statistical heterogeneity, which causes a drift in the local updates that negatively impacts convergence. We propose a new algorithm VR-FALD* that uses control variates to correct the client drift. We establish non-asymptotic bounds showing that VR-FALD* is not affected by statistical heterogeneity. Finally, we illustrate our results on several FL benchmarks for Bayesian inference.

Dates et versions

hal-04396699 , version 1 (16-01-2024)

Identifiants

Citer

Vincent Plassier, Alain Durmus, Eric Moulines. Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, 2023. ⟨hal-04396699⟩
8 Consultations
0 Téléchargements

Altmetric

Partager

More