The bane of skew: Uncertain ranks and unrepresentative precision - Ecole Nationale du Génie de l'Eau et de l'Environnement de Strasbourg Accéder directement au contenu
Article Dans Une Revue Machine Learning Année : 2014

The bane of skew: Uncertain ranks and unrepresentative precision

Pierre Gançarski

Résumé

While a problem's skew is often assumed to be constant, this paper discusses three settings where this assumption does not hold. Consequently, incorrectly assuming skew to be constant in these contradicting cases results in an over or under estimation of an algorithm's performance. The area under a precision-recall curve (AUCPR) is a common summary measurement used to report the performance of machine learning algorithms. It is well known that precision is dependent upon class skew, which often varies between datasets. In addition to this, it is demonstrated herein that under certain circumstances the relative ranking of an algorithm (as measured by AUCPR) is not constant and is instead also dependent upon skew. The skew at which the performance of two algorithms inverts and the relationship between precision measured at different skews are defined. This is extended to account for temporal skew characteristics and situations in which skew cannot be precisely defined. Formal proofs for these findings are presented, desirable properties are proved and their application demonstrated.
Fichier principal
Vignette du fichier
Lampert14TheBaneOfSkew.pdf (2.63 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03175422 , version 1 (20-03-2021)

Identifiants

Citer

Thomas A Lampert, Pierre Gançarski. The bane of skew: Uncertain ranks and unrepresentative precision. Machine Learning, 2014, 97 (1-2), pp.5-32. ⟨10.1007/s10994-013-5432-x⟩. ⟨hal-03175422⟩
30 Consultations
21 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More