Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data

Résumé

For classification and regression on tabular data, the dominance of gradient-boosted decision trees (GBDTs) has recently been challenged by often much slower deep learning methods with extensive hyperparameter tuning. We address this discrepancy by introducing (a) RealMLP, an improved multilayer perceptron (MLP), and (b) improved default parameters for GBDTs and RealMLP. We tune RealMLP and the default parameters on a meta-train benchmark with 71 classification and 47 regression datasets and compare them to hyperparameter-optimized versions on a disjoint meta-test benchmark with 48 classification and 42 regression datasets, as well as the GBDT-friendly benchmark by Grinsztajn et al. (2022). Our benchmark results show that RealMLP offers a better time-accuracy tradeoff than other neural nets and is competitive with GBDTs. Moreover, a combination of RealMLP and GBDTs with improved default parameters can achieve excellent results on medium-sized tabular datasets (1K--500K samples) without hyperparameter tuning.
Fichier principal
Vignette du fichier
paper_arxiv_without_comments.pdf (2.25 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04641923 , version 1 (09-07-2024)

Licence

Identifiants

Citer

David Holzmüller, Léo Grinsztajn, Ingo Steinwart. Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data. 2024. ⟨hal-04641923⟩
14 Consultations
6 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More