Neural Architecture Search: A Visual Analysis - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Neural Architecture Search: A Visual Analysis

Gabriela Ochoa

Résumé

Neural architecture search (NAS) refers to the use of search heuristics to optimise the topology of deep neural networks. NAS algorithms have produced topologies that outperform human-designed ones. However, contrasting alternative NAS methods is difficult. To address this, several tabular NAS benchmarks have been proposed that exhaustively evaluate all architectures in a given search space. We conduct a thorough fitness landscape analysis of a popular tabular, cell-based NAS benchmark. Our results indicate that NAS landscapes are multi-modal, but have a relatively low number of local optima, from which it is not hard to escape. We confirm that reducing the noise in estimating performance reduces the number of local optima. We hypothesise that localsearch based NAS methods are likely to be competitive, which we confirm by implementing a landscape-aware iterated local search algorithm that can outperform more elaborate evolutionary and reinforcement learning NAS methods.
Fichier principal
Vignette du fichier
PPSN_2022_NAS_Landscapes.pdf (1.05 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03822771 , version 1 (20-10-2022)

Identifiants

Citer

Gabriela Ochoa, Nadarajen Veerapen. Neural Architecture Search: A Visual Analysis. 17th International Conference on Parallel Problem Solving from Nature (PPSN XVII), Sep 2022, Dortmund, Germany. pp.603-615, ⟨10.1007/978-3-031-14714-2_42⟩. ⟨hal-03822771⟩
18 Consultations
143 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More