Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statistics - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statistics

Pascal Pernot

Résumé

Abstract Post hoc recalibration of prediction uncertainties of machine learning regression problems by isotonic regression might present a problem for bin-based calibration error statistics (e.g. ENCE). Isotonic regression often produces stratified uncertainties, i.e. subsets of uncertainties with identical numerical values. Partitioning of the resulting data into equal-sized bins introduces an aleatoric component to the estimation of bin-based calibration statistics. The partitioning of stratified data into bins depends on the order of the data, which is typically an uncontrolled property of calibration test/validation sets. The tie-braking method of the ordering algorithm used for binning might also introduce an aleatoric component. I show on an example how this might significantly affect the calibration diagnostics.

Domaines

Chimie

Dates et versions

hal-04131182 , version 1 (16-06-2023)

Identifiants

Citer

Pascal Pernot. Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statistics. 2023. ⟨hal-04131182⟩
8 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More