MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2015

MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization

Résumé

In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities. We propose this computational model as an answer to the question of how some class of concepts can be learnt. The model is also a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to lower the ambiguity of learnt concepts as well as communicate about them. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motion acquisitions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts to reproduce the experiments are publicly available.
Fichier principal
Vignette du fichier
multimodal.pdf (881.39 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01137529 , version 1 (30-03-2015)
hal-01137529 , version 2 (04-05-2015)
hal-01137529 , version 3 (29-10-2015)

Licence

Paternité - Pas d'utilisation commerciale

Identifiants

  • HAL Id : hal-01137529 , version 2

Citer

Olivier Mangin, David Filliat, Pierre-Yves Oudeyer. MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization. 2015. ⟨hal-01137529v2⟩
330 Consultations
234 Téléchargements

Partager

Gmail Facebook X LinkedIn More