Pré-Publication, Document De Travail Année : 2025

How good is PAC-Bayes at explaining generalisation?

Résumé

We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced by the prior distribution. In particular, achieving a target generalisation level is only achievable if the prior places sufficient mass on high-performing predictors. We relate these requirements to the prevalent practice of using data-dependent priors in deep learning PAC-Bayes applications, and discuss the implications for the claim that PAC-Bayes ``explains'' generalisation.

Dates et versions

hal-04989460 , version 1 (13-03-2025)

Identifiants

Citer

Antoine Picard-Weibel, Eugenio Clerico, Roman Moscoviz, Benjamin Guedj. How good is PAC-Bayes at explaining generalisation?. 2025. ⟨hal-04989460⟩
126 Consultations
0 Téléchargements

Altmetric

Partager

  • More