A Bandit Approach with Evolutionary Operators for Model Selection - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

A Bandit Approach with Evolutionary Operators for Model Selection

Résumé

This paper formulates model selection as an infinite-armed bandit problem. The models are arms, and picking an arm corresponds to a partial training of the model (resource allocation). The reward is the accuracy of the selected model after its partial training. In this best arm identification problem, regret is the gap between the expected accuracy of the optimal model and that of the model finally chosen. We first consider a straightforward generalization of UCB-E to the stochastic infinite-armed bandit problem and show that, under basic assumptions, the expected regret order is $T^{-\alpha}$ for some $\alpha \in (0,1/5)$ and $T$ the number of resources to allocate. From this vanilla algorithm, we introduce the algorithm Mutant-UCB that incorporates operators from evolutionary algorithms. Tests carried out on three open source image classification data sets attest to the relevance of this novel combining approach, which outperforms the state-of-the-art for a fixed budget.
Fichier principal
Vignette du fichier
main.pdf (572.99 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04440552 , version 1 (06-02-2024)

Identifiants

Citer

Margaux Brégère, Julie Keisler. A Bandit Approach with Evolutionary Operators for Model Selection: Application to Neural Architecture Optimization for Image Classification. 2024. ⟨hal-04440552⟩
35 Consultations
19 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More