Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue APL Machine Learning Année : 2023

Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity

Pascal Pernot

Résumé

Reliable uncertainty quantification (UQ) in machine learning (ML) regression tasks is becoming the focus of many studies in materials and chemical science. It is now well understood that average calibration is insufficient, and most studies implement additional methods testing the conditional calibration with respect to uncertainty, i.e. consistency. Consistency is assessed mostly by so-called reliability diagrams. There exists however another way beyond average calibration, which is conditional calibra- tion with respect to input features, i.e. adaptivity. In practice, adaptivity is the main concern of the final users of a ML-UQ method, seeking for the reliability of predic- tions and uncertainties for any point in features space. This article aims to show that consistency and adaptivity are complementary validation targets, and that a good consistency does not imply a good adaptivity. An integrated validation framework is proposed and illustrated on a representative example.
Fichier principal
Vignette du fichier
Pernot2023c.pdf (2.43 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY ND - Paternité - Pas de modifications

Dates et versions

hal-04356591 , version 1 (20-12-2023)

Identifiants

Citer

Pascal Pernot. Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity. APL Machine Learning, 2023, 1 (4), pp.046121. ⟨10.1063/5.0174943⟩. ⟨hal-04356591⟩
26 Consultations
32 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More