Sensitivity Analysis of RF+clust for Leave-One-Problem-Out Performance Prediction - Archive ouverte HAL
Communication Dans Un Congrès Année : 2023

Sensitivity Analysis of RF+clust for Leave-One-Problem-Out Performance Prediction

Ana Nikolikj
  • Fonction : Auteur
Michal Pluháček
  • Fonction : Auteur
Carola Doerr
Peter Korošec
  • Fonction : Auteur
Tome Eftimov
  • Fonction : Auteur

Résumé

Leave-one-problem-out (LOPO) performance prediction requires machine learning (ML) models to extrapolate algorithms' performance from a set of training problems to a previously unseen problem. LOPO is a very challenging task even for state-of-the-art approaches. Models that work well in the easier leave-one-instance-out scenario often fail to generalize well to the LOPO setting. To address the LOPO problem, recent work suggested enriching standard random forest (RF) performance regression models with a weighted average of algorithms' performance on training problems that are considered similar to a test problem. More precisely, in this RF+clust approach, the weights are chosen proportionally to the distances of the problems in some feature space. Here in this work, we extend the RF+clust approach by adjusting the distance-based weights with the importance of the features for performance regression. That is, instead of considering cosine distance in the feature space, we consider a weighted distance measure, with weights depending on the relevance of the feature for the regression model. Our empirical evaluation of the modified RF+clust approach on the CEC 2014 benchmark suite confirms its advantages over the naive distance measure. However, we also observe room for improvement, in particular with respect to more expressive feature portfolios.
Fichier principal
Vignette du fichier
CEC_RF_Clust.pdf (4.53 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04242051 , version 1 (14-10-2023)

Identifiants

Citer

Ana Nikolikj, Michal Pluháček, Carola Doerr, Peter Korošec, Tome Eftimov. Sensitivity Analysis of RF+clust for Leave-One-Problem-Out Performance Prediction. 2023 IEEE Congress on Evolutionary Computation (CEC), Jul 2023, Chicago, IL, United States. pp.1-8, ⟨10.1109/CEC53210.2023.10254146⟩. ⟨hal-04242051⟩
35 Consultations
28 Téléchargements

Altmetric

Partager

More