Forecast evaluation with imperfect observations and imperfect models - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Forecast evaluation with imperfect observations and imperfect models

Julie Bessac
  • Fonction : Auteur
  • PersonId : 772563
  • IdRef : 182856798

Résumé

The field of statistics has become one of the mathematical foundations in forecast evaluations studies, especially in regard to computing scoring rules. The classical paradigm of proper scoring rules is to discriminate between two different forecasts by comparing them with observations. The probability density function of the observed record is assumed to be perfect as a verification benchmark. In practice, however, observations are almost always tainted by errors. These may be due to homogenization problems, instrumental deficiencies, the need for indirect reconstructions from other sources (e.g., radar data), model errors in gridded products like reanalysis, or any other data-recording issues. If the yardstick used to compare forecasts is imprecise, one can wonder whether such types of errors may or may not have a strong influence on decisions based on classical scoring rules. Building on the recent work of Ferro (2017), we propose a new scoring rule scheme in the context of models that incorporate errors of the verification data, we compare it to existing methods, and applied it to various setups, mainly a Gaussian additive noise model and a gamma multiplicative noise model. In addition, we frame the problem of error verification in datasets as scoring a model that jointly couples forecasts and observation distributions. This is strongly connected to the so-called error-in-variables models in statistics.

Dates et versions

hal-03057740 , version 1 (11-12-2020)

Identifiants

Citer

Philippe Naveau, Julie Bessac. Forecast evaluation with imperfect observations and imperfect models. 2020. ⟨hal-03057740⟩
44 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More