LS-PON: a Prediction-based Local Search for Neural Architecture Search - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

LS-PON: a Prediction-based Local Search for Neural Architecture Search

Résumé

Neural architecture search (NAS) is a subdomain of AutoML that consists of automating the design of neural networks. NAS has become a hot topic in the last few years. As a result, many methods are being developed in this area. Local search (LS), on the other hand, is a famous heuristic that has been around for many years. It is extensively used for optimization problems due to its simplicity and efficiency. LS has a lot of advantages in the world of NAS; it can naturally exploit methods that accelerate the global search time such as weight inheritance and network morphism. LS is also easy to implement and does not require a complex encoding or any parameter tuning. In the present work, we aim at making LS faster by guiding the exploration of the neighborhood. Our objective is to limit the number of solution evaluations, which are particularly time-consuming in NAS. We propose the method LS-PON (Local Search with a Predicted Order of Neighbors) that uses linear regression models to order the exploration of neighbors during the search. LS-PON, unlike other prediction-based NAS methods, requires neither pre-sampling nor tuning. Our experiments on popular NAS benchmarks show that LS-PON keeps the simplicity and advantages of LS while being as efficient in quality as state-of-the-art methods and can be more than twice as fast as classical LS.
Fichier principal
Vignette du fichier
LS_PON_LOD2022_Camera_Ready (1).pdf (1.92 Mo) Télécharger le fichier
LS-PON_LOD2022_Camera_Ready.zip (3.22 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03797130 , version 1 (05-04-2023)

Identifiants

  • HAL Id : hal-03797130 , version 1

Citer

Meyssa Zouambi, Julie Jacques, Clarisse Dhaenens. LS-PON: a Prediction-based Local Search for Neural Architecture Search. The 8th Annual Conference on machine Learning, Optimization and Data science LOD2022, Sep 2022, Siena, Italy. ⟨hal-03797130⟩
32 Consultations
27 Téléchargements

Partager

Gmail Facebook X LinkedIn More