Discussion on Minimal Penalty and the Slope Heuristic: A Survey, by S. Arlot
Résumé
This paper gives a very complete overview of the state of the art on minimal penalties and slope heuristics developed these past 20 years. Sylvain Arlot has to be congratulated for providing a survey which clearly relies theory and heuristics, and which proposes several ways to proceed heuristics in practice. My comments will focus on binary classification with the 0-1 loss, for which, as mentioned several times by Sylvain Arlot, there is still no clear results on slope heuristics. We propose to illustrate the behavior of two slope heuristics by an empirical simulation study, and on the benchmark well-known spam data set. We focus on classification trees model selection with the 0-1 loss, and we use the CART algorithm proposed by Breiman et al. in 1984 [3] to build the models. The results of the slope heuristics are compared with the ones obtained by the classical 10-fold cross-validation via the prediction errors of the selected classifiers, and the dimensions of the corresponding classification tree models.
Domaines
Statistiques [math.ST]Origine | Fichiers produits par l'(les) auteur(s) |
---|
Loading...