Entropy of Overcomplete Kernel Dictionaries - Archive ouverte HAL
Article Dans Une Revue Bulletin of Mathematical Sciences and Applications Année : 2016

Entropy of Overcomplete Kernel Dictionaries

Paul Honeine

Résumé

In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the orthogonality condition of the atoms, yielding overcomplete dictionaries with an extended number of atoms. More generally than the linear decomposition, overcomplete kernel dictionaries provide an elegant nonlinear extension by defining the atoms through a mapping kernel function (e.g., the gaussian kernel). Models based on such kernel dictionaries are used in neural networks, gaussian processes and online learning with kernels. The quality of an overcomplete dictionary is evaluated with a diversity measure the distance, the approximation, the coherence and the Babel measures. In this paper, we develop a framework to examine overcomplete kernel dictionaries with the entropy from information theory. Indeed, a higher value of the entropy is associated to a further uniform spread of the atoms over the space. For each of the aforementioned diversity measures, we derive lower bounds on the entropy. Several definitions of the entropy are examined, with an extensive analysis in both the input space and the mapped feature space.
Fichier principal
Vignette du fichier
16.entropy.pdf (222.48 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01965563 , version 1 (26-12-2018)

Identifiants

  • HAL Id : hal-01965563 , version 1

Citer

Paul Honeine. Entropy of Overcomplete Kernel Dictionaries. Bulletin of Mathematical Sciences and Applications, 2016, 16, pp.1 - 19. ⟨hal-01965563⟩
31 Consultations
31 Téléchargements

Partager

More