D-DARTS: Distributed Differentiable Architecture Search - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2021

D-DARTS: Distributed Differentiable Architecture Search

Résumé

Differentiable ARchiTecture Search (DARTS) is one of the most trending Neural Architecture Search (NAS) methods, drastically reducing search cost by resorting to Stochastic Gradient Descent (SGD) and weight-sharing. However, it also greatly reduces the search space, thus excluding potential promising architectures from being discovered. In this paper, we propose D-DARTS, a novel solution that addresses this problem by nesting several neural networks at cell-level instead of using weight-sharing to produce more diversified and specialized architectures. Moreover, we introduce a novel algorithm which can derive deeper architectures from a few trained cells, increasing performance and saving computation time. Our solution is able to provide state-of-the-art results on CIFAR-10, CIFAR-100 and ImageNet while using significantly less parameters than previous baselines, resulting in more hardware-efficient neural networks.

Dates et versions

hal-03324857 , version 1 (24-08-2021)

Identifiants

Citer

Alexandre Heuillet, Hedi Tabia, Hichem Arioui, Kamal Youcef-Toumi. D-DARTS: Distributed Differentiable Architecture Search. 2021. ⟨hal-03324857⟩
47 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More