Robust PACm: Training Ensemble Models Under Misspecification and Outliers - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Neural Networks and Learning Systems Année : 2023

Robust PACm: Training Ensemble Models Under Misspecification and Outliers

Matteo Zecchin
Sangwoo Park
Osvaldo Simeone

Résumé

Standard Bayesian learning is known to have suboptimal generalization capabilities under misspecification and in the presence of outliers. Probably approximately correct (PAC)-Bayes theory demonstrates that the free energy criterion minimized by Bayesian learning is a bound on the generalization error for Gibbs predictors (i.e., for single models drawn at random from the posterior) under the assumption of sampling distributions uncontaminated by outliers. This viewpoint provides a justification for the limitations of Bayesian learning when the model is misspecified, requiring ensembling, and when data are affected by outliers. In recent work, PAC-Bayes bounds—referred to as PAC m —were derived to introduce free energy metrics that account for the performance of ensemble predictors, obtaining enhanced performance under misspecification. This work presents a novel robust free energy criterion that combines the generalized logarithm score function with PAC m ensemble bounds. The proposed free energy training criterion produces predictive distributions that are able to concurrently counteract the detrimental effects of misspecification—with respect to both likelihood and prior distribution—and outliers.
Fichier principal
Vignette du fichier
Robust_PAC_m__Training_Ensemble_Models_Under_Misspecification_and_Outliers.pdf (2.65 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04182182 , version 1 (21-08-2023)

Identifiants

Citer

Matteo Zecchin, Sangwoo Park, Osvaldo Simeone, Marios Kountouris, David Gesbert. Robust PACm: Training Ensemble Models Under Misspecification and Outliers. IEEE Transactions on Neural Networks and Learning Systems, 2023, pp.1-15. ⟨10.1109/TNNLS.2023.3295168⟩. ⟨hal-04182182⟩
26 Consultations
9 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More