Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion

Résumé

For interactivity and cost-efficiency purposes, both biological and artificial agents (e.g., robots) usually rely on sets of complementary sensors. Each sensor samples information from only a subset of the environment, with both the subset and the precision of signals varying through time depending on the agent-environment configuration. Agents must therefore perform multimodal fusion to select and filter relevant information by contrasting the shortcomings and redundancies of different modalities. For that purpose, we propose to combine a classical off-the-shelf manifold learning algorithm with dynamic neural fields (DNF), a training-free bio-inspired model of competition amid topologically-encoded information. Through the adaptation of DNF to irregular multimodal topologies, this coupling exhibits interesting properties, promising reliable localizations enhanced by the selection and attentional capabilities of DNF. In particular, the application of our method to audiovisual datasets (with direct ties to either psychophysics or robotics) shows merged perceptions relying on the spatially-dependent precision of each modality, and robustness to irrelevant features.
Fichier principal
Vignette du fichier
main_hal.pdf (2.12 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03693198 , version 1 (10-06-2022)

Identifiants

Citer

Simon Forest, Jean-Charles Quinton, Mathieu Lefort. Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion. 2022 International Joint Conference on Neural Networks (IJCNN), Jul 2022, Padua, Italy. ⟨10.1109/IJCNN55064.2022.9892614⟩. ⟨hal-03693198⟩
91 Consultations
166 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More