Channel Configuration for Neural Architecture: Insights from the Search Space - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Channel Configuration for Neural Architecture: Insights from the Search Space

Sarah L Thomson
  • Fonction : Auteur
  • PersonId : 1033744
Gabriela Ochoa
Krzysztof Michalak
  • Fonction : Auteur
  • PersonId : 1252506

Résumé

We consider search spaces associated with neural network channel configuration. Architectures and their accuracy are visualised using low-dimensional Euclidean embedding (LDEE). Optimisation dynamics are captured using local optima networks (LONs). LONs are a compression of a fitness landscape: the nodes are local optima and the edges are search transitions between them. Several neural architecture search algorithms are tested on the search space and we discover that iterated local search (ILS) is a competitive algorithm for neural channel configuration. We additionally implement a landscape-aware ILS which performs well. Observations from the search and landscape space analyses bring visual clarity and insight to the science of neural network channel design: the results indicate that a high number of channels, kept constant throughout the network, is beneficial.
Fichier principal
Vignette du fichier
GECCO_2023__NAS_LONs__authorversion.pdf (5.95 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04090650 , version 1 (05-05-2023)

Identifiants

Citer

Sarah L Thomson, Gabriela Ochoa, Nadarajen Veerapen, Krzysztof Michalak. Channel Configuration for Neural Architecture: Insights from the Search Space. GECCO ’23: Genetic and Evolutionary Computation Conference, ACM, Jul 2023, Lisbonne, Portugal. ⟨10.1145/3583131.3590386⟩. ⟨hal-04090650⟩
17 Consultations
5 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More