Categorical Perception: A Groundwork for Deep Learning - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Categorical Perception: A Groundwork for Deep Learning

Résumé

Classification is one of the major tasks that deep learning is successfully tackling. Categorization is also a fundamental cognitive ability. A well-known perceptual consequence of categorization in humans and other animals, called categorical perception, is characterized by a within-category compression and a between-category separation: two items, close in input space, are perceived closer if they belong to the same category than if they belong to different categories. Elaborating on experimental and theoretical results in cognitive science, here we study categorical effects in artificial neural networks. Our formal and numerical analysis provides insights into the geometry of the neural representation in deep layers, with expansion of space near category boundaries and contraction far from category boundaries. We investigate categorical representation by using two complementary approaches: one mimics experiments in psychophysics and cognitive neuroscience by means of morphed continua between stimuli of different categories, while the other introduces a categoricality index that quantifies the separability of the classes at the population level (a given layer in the neural network). We show on both shallow and deep neural networks that category learning automatically induces categorical perception. We further show that the deeper a layer, the stronger the categorical effects. An important outcome of our analysis is to provide a coherent and unifying view of the efficacy of different heuristic practices of the dropout regularization technique. Our views, which find echoes in the neuroscience literature, insist on the differential role of noise as a function of the level of representation and in the course of learning: noise injected in the hidden layers gets structured according to the organization of the categories, more variability being allowed within a category than across classes.
Fichier principal
Vignette du fichier
categorical_perception_deep_learning_preprint.pdf (5.19 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03053998 , version 1 (11-12-2020)
hal-03053998 , version 2 (15-11-2021)

Identifiants

Citer

Laurent Bonnasse-Gahot, Jean-Pierre Nadal. Categorical Perception: A Groundwork for Deep Learning. 2020. ⟨hal-03053998v1⟩
258 Consultations
72 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More