Online Task-free Continual Learning with Dynamic Sparse Distributed Memory - Archive ouverte HAL
Communication Dans Un Congrès Année : 2022

Online Task-free Continual Learning with Dynamic Sparse Distributed Memory

Résumé

This paper addresses the very challenging problem of online task-free continual learning in which a sequence of new tasks is learned from non-stationary data using each sample only once for training and without knowledge of task boundaries. We propose in this paper an efficient semi-distributed associative memory algorithm called Dynamic Sparse Distributed Memory (DSDM) where learning and evaluating can be carried out at any point of time. DSDM evolves dynamically and continually modeling the distribution of any non-stationary data stream. DSDM relies on locally distributed, but only partially overlapping clusters of representations to effectively eliminate catastrophic forgetting, while at the same time, maintaining the generalization capacities of distributed networks. In addition, a local density-based pruning technique is used to control the network's memory footprint. DSDM significantly outperforms state-of-the-art continual learning methods on different image classification baselines, even in a low data regime.
Fichier principal
Vignette du fichier
136850721.pdf (693.24 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04473476 , version 1 (22-02-2024)

Identifiants

Citer

Julien Pourcel, Ngoc-Son Vu, Robert M. French. Online Task-free Continual Learning with Dynamic Sparse Distributed Memory. 2022 European Conference on Computer Vision (EECV 2022), Oct 2022, Tel Aviv, Israel. ⟨10.1007/978-3-031-19806-9_42⟩. ⟨hal-04473476⟩
121 Consultations
176 Téléchargements

Altmetric

Partager

More