Confidence curves for UQ validation: probabilistic reference vs. oracle - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Confidence curves for UQ validation: probabilistic reference vs. oracle

Pascal Pernot

Résumé

Confidence curves are used in uncertainty validation to assess how large uncertainties ($u_{E}$) are associated with large errors ($E$). An oracle curve is commonly used as reference to estimate the quality of the tested datasets. The oracle is a perfect, deterministic, error predictor, such as $|E|=\pm u_{E}$, which corresponds to a very unlikely error distribution in a probabilistic framework and is unable unable to inform us on the calibration of $u_{E}$. I propose here to replace the oracle by a probabilistic reference curve, deriving from the more realistic scenario where errors should be random draws from a distribution with standard deviation $u_{E}$. The probabilistic curve and its confidence interval enable a direct test of the quality of a confidence curve. Paired with the probabilistic reference, a confidence curve can be used to check the calibration and tightness of prediction uncertainties.

Dates et versions

hal-04131189 , version 1 (16-06-2023)

Identifiants

Citer

Pascal Pernot. Confidence curves for UQ validation: probabilistic reference vs. oracle. 2023. ⟨hal-04131189⟩
11 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More