Evaluating machine learning models and their diagnostic value - Archive ouverte HAL Accéder directement au contenu
Chapitre D'ouvrage Année : 2022

Evaluating machine learning models and their diagnostic value

Résumé

This chapter describes how to validate a machine learning model. We start by detailing the main performance metrics for different tasks (classification, regression), and how they may be interpreted, including in the face of class imbalance, varying prevalence, or asymmetric cost-benefit trade-offs. We then explain how to estimate these metrics in a unbiased manner using training, validation, and test sets. We describe crossvalidation procedures-to use a larger part of the data for both training and testing-and the dangers of data leakage-optimism bias due to training data contaminating the test set. Finally, we discuss how to obtain confidence intervals of performance metrics, distinguishing two situations: internal validation or evaluation of learning algorithms, and external validation or evaluation of resulting prediction models.
Fichier principal
Vignette du fichier
main.pdf (391.12 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03682454 , version 1 (31-05-2022)
hal-03682454 , version 2 (01-06-2022)
hal-03682454 , version 3 (02-06-2022)
hal-03682454 , version 4 (21-01-2023)
hal-03682454 , version 5 (20-04-2023)

Identifiants

  • HAL Id : hal-03682454 , version 1

Citer

Gaël Varoquaux, Olivier Colliot. Evaluating machine learning models and their diagnostic value. Machine Learning for Brain Disorders, In press. ⟨hal-03682454v1⟩
11653 Consultations
3101 Téléchargements

Partager

Gmail Facebook X LinkedIn More