Finding Concept Representations in Neural Networks with Self-Organizing Maps - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Finding Concept Representations in Neural Networks with Self-Organizing Maps

Résumé

In sufficiently complex tasks, it is expected that as a side effect of learning to solve a problem, a neural network will learn relevant abstractions of the representation of that problem. This has been confirmed in particular in machine vision where a number of works showed that correlations could be found between the activations of specific units (neurons) in a neural network and the visual concepts (textures, colors, objects) present in the image. Here, we explore the use of self-organizing maps as a way to both visually and computationally inspect how activation vectors of whole layers of neural networks correspond to neural representations of abstract concepts such as 'female person' or 'realist painter'. We experiment with multiple measures applied to those maps to assess the level of representation of a concept in a network's layer. We show that, among the measures tested, the relative entropy of the activation map for a concept compared to the map for the whole data is a suitable candidate and can be used as part of a methodology to identify and locate the neural representation of a concept, visualize it, and understand its importance in solving the prediction task at hand. CCS CONCEPTS • Computing methodologies → Artificial intelligence; Neural networks; Knowledge representation and reasoning.
Fichier principal
Vignette du fichier
K_CAP2023_SOM (5).pdf (979.53 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04315074 , version 1 (30-11-2023)

Identifiants

Citer

Mathieu D’aquin. Finding Concept Representations in Neural Networks with Self-Organizing Maps. K-CAP '23: Knowledge Capture Conference 2023, Dec 2023, Pensacola FL USA, United States. pp.53-60, ⟨10.1145/3587259.3627551⟩. ⟨hal-04315074⟩
45 Consultations
6 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More