DARTS with Degeneracy Correction - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

DARTS with Degeneracy Correction

Résumé

The neural architecture search (NAS) is characterized by a wide search space and a time consuming objective function. Many papers have dealt with the reduction of the cost of the objective function assessment. Among them, there is DARTS paper [1] that proposes to transform the original discrete problem into a continuous one. This paper builds an overparameterized network called hypernetwork and weights its edges by continuous coefficients. This approach allows to considerably reduce the computational cost, but the quality of the obtained architectures is highly variable. We propose to reduce this variability by introducing a convex depth regularization. We also add a heuristic that controls the number of unweighted operations. The goal is to correct the short term bias, introduced by the hypergradient approximation. Finally, we will show the efficiency of these proposals by starting again the work developed in Dots paper [2].
Fichier non déposé

Dates et versions

hal-04360527 , version 1 (21-12-2023)

Identifiants

Citer

Guillaume Lacharme, Hubert Cardot, Christophe Lenté, Nicolas Monmarché. DARTS with Degeneracy Correction. Pattern Recognition and Image Analysis: 11th Iberian Conference, IbPRIA 2023, Jun 2023, Alicante, Spain. pp.40-53, ⟨10.1007/978-3-031-36616-1_4⟩. ⟨hal-04360527⟩
25 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More