Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion

Abstract

For interactivity and cost-efficiency purposes, both biological and artificial agents (e.g., robots) usually rely on sets of complementary sensors. Each sensor samples information from only a subset of the environment, with both the subset and the precision of signals varying through time depending on the agent-environment configuration. Agents must therefore perform multimodal fusion to select and filter relevant information by contrasting the shortcomings and redundancies of different modalities. For that purpose, we propose to combine a classical off-the-shelf manifold learning algorithm with dynamic neural fields (DNF), a training-free bio-inspired model of competition amid topologically-encoded information. Through the adaptation of DNF to irregular multimodal topologies, this coupling exhibits interesting properties, promising reliable localizations enhanced by the selection and attentional capabilities of DNF. In particular, the application of our method to audiovisual datasets (with direct ties to either psychophysics or robotics) shows merged perceptions relying on the spatially-dependent precision of each modality, and robustness to irrelevant features.
Fichier principal
Vignette du fichier
main_hal.pdf (2.12 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03693198 , version 1 (10-06-2022)

Identifiers

Cite

Simon Forest, Jean-Charles Quinton, Mathieu Lefort. Combining Manifold Learning and Neural Field Dynamics for Multimodal Fusion. 2022 International Joint Conference on Neural Networks (IJCNN), Jul 2022, Padua, Italy. ⟨10.1109/IJCNN55064.2022.9892614⟩. ⟨hal-03693198⟩
103 View
189 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More