Cortex Inspired Learning to Recover Damaged Signal Modality by ReD-SOM Model - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Cortex Inspired Learning to Recover Damaged Signal Modality by ReD-SOM Model

Résumé

Recent progress in the fields of AI and cognitive sciences opens up new challenges and problems that were previously inaccessible to study. One of such modern tasks is recovering lost data of one modality by using the data from another one. A similar effect (called the McGurk Effect) has been found in the functioning of the human brain. Observing this effect, one modality of information interferes with another, changing its perception. In this paper, we propose a way to reproduce such an effect and use it to reconstruct lost data modalities by combining Variational Auto-Encoders, Self-Organizing Maps, and Hebb connections in a unified ReD-SOM (Reentering Deep Self-organizing Map) model. We are inspired by human's capability to use different zones of the brain in different modalities, in case of having a lack of information in one of the modalities. This new approach not only improves the analysis of ambiguous data but also restores the intended signal. The results obtained on the multimodal dataset show an increase of quality of the signal reconstruction. The effect is remarkable both visually and quantitatively, specifically in presence of a significant degree of signal's distortion.

Dates et versions

hal-04141988 , version 1 (26-06-2023)

Identifiants

Citer

Artem Muliukov, Laurent Rodriguez, Benoit Miramond. Cortex Inspired Learning to Recover Damaged Signal Modality by ReD-SOM Model. IJCNN 2023, IEEE, Jun 2023, Gold Coast, Australia, Australia. pp.01-09, ⟨10.1109/IJCNN54540.2023.10191701⟩. ⟨hal-04141988⟩
22 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More