Binary Classifier Evaluation Without Ground Truth - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2017

Binary Classifier Evaluation Without Ground Truth

Résumé

In this paper we study statistically sound ways of comparing classifiers in absence for fully reliable reference data. Based on previously published partial frameworks, we explore a more comprehensive approach to comparing and ranking classifiers that is robust to incomplete, erroneous or missing reference evaluation data. On the one hand, the use of a generalized McNemar's test is shown to give reliable confidence measures in the ranking of two classifiers under the assumption of an existing better-than-random reference classifier. We extend its use to cases where its traditional formulation is notoriously unstable. We also provide a computational context that allows it to be used for large amounts of data. Our classifier evaluation model is generic and applies to any set of binary classifiers. We have more specifically tested and validated it on synthetic and real data coming from document image binarization.
Fichier principal
Vignette du fichier
icapr2017.pdf (258.08 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01680358 , version 1 (10-01-2018)

Identifiants

  • HAL Id : hal-01680358 , version 1

Citer

Maksym Fedorchuk, Bart Lamiroy. Binary Classifier Evaluation Without Ground Truth. Ninth International Conference on Advances in Pattern Recognition (ICAPR-2017), Dec 2017, Bangalore, India. ⟨hal-01680358⟩
297 Consultations
2534 Téléchargements

Partager

Gmail Facebook X LinkedIn More