Learning the Structure of Deep Architectures Using L1 Regularization - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2015

Learning the Structure of Deep Architectures Using L1 Regularization

Praveen Kulkarni
  • Fonction : Auteur
  • PersonId : 975955
Joaquin Zepeda
  • Fonction : Auteur
  • PersonId : 883135
Patrick Pérez
  • Fonction : Auteur
  • PersonId : 1022281
Louis Chevallier
  • Fonction : Auteur
  • PersonId : 940855

Résumé

We present a method that formulates the selection of the structure of a deep architecture as a penalized, discriminative learning problem. Up to now, the structure of deep architectures has been fixed by hand, and only the weights are learned using discriminative learning. Our work is a first attempt towards a more formal method of deep structure selection. We consider architectures consisting only of fully-connected layers, and our approach relies on diagonal matrices inserted between subsequent layers. By including an L1 norm of the diagonal entries of said matrices as a regularization penalty, we force the diagonals to be sparse, accordingly selecting the effective number of rows (respectively, columns) of the corresponding layer's (next layer's) weights matrix. We carry out experiments on a standard dataset and show that our method succeeds in selecting the structure of deep architectures of multiple layers. One variant of our architecture results in a feature vector of size as little as $36$, while retaining very high image classification performance.
Fichier principal
Vignette du fichier
hal-01266462.pdf (261.65 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01266462 , version 1 (13-10-2017)

Identifiants

Citer

Praveen Kulkarni, Joaquin Zepeda, Frédéric Jurie, Patrick Pérez, Louis Chevallier. Learning the Structure of Deep Architectures Using L1 Regularization. British Machine Vision Conference, 2015, Sep 2015, swansea, United Kingdom. ⟨10.5244/C.29.23⟩. ⟨hal-01266462⟩
272 Consultations
282 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More