A Multi-objective Model Search Algorithm for Linear Regression - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2023

A Multi-objective Model Search Algorithm for Linear Regression

Résumé

Inherent in model selection is the problem of simultaneously optimizing multiple performance metrics. Some of these metrics express potentially conflicting criteria, like accuracy and simplicity. Pareto optimization is a branch of mathematical optimization that deals precisely with problems involving conflicting objective functions. In this article, an algorithm was developed that searches automatically for Pareto optimal linear regression models given a dataset and a set of performance metrics. The optimization task was framed as one of sequential variable selection on a graph. A search strategy was proposed that draws on ant colony optimization, a probabilistic technique well suited for graph-based problems. Experiments were run in which the metrics to be minimized were the root-mean-square error, expressing accuracy, and the number of coefficients, expressing simplicity. To substantiate the usefulness of our algorithm, cases were presented in which it outperformed AIC-based stepwise regression. Results suggested that our algorithm copes well with small datasets and correlated predictors, that it is efficient and that it informs model selection. Key properties of our algorithm were discussed and areas of improvement highlighted.
Fichier principal
Vignette du fichier
Paper___JCGS_version-17.pdf (310.54 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04101559 , version 1 (20-05-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04101559 , version 1

Citer

Anas Mifrani, Philippe Saint-Pierre, Nicolas Savy. A Multi-objective Model Search Algorithm for Linear Regression. 2023. ⟨hal-04101559⟩
36 Consultations
62 Téléchargements

Partager

Gmail Facebook X LinkedIn More