Adaptive Neural Networks for Online Domain Incremental Continual Learning - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Adaptive Neural Networks for Online Domain Incremental Continual Learning

Résumé

Continual Learning (CL) poses a significant challenge to Neural Network (NN)s, where the data distribution changes from one task to another. In Online domain incremental continual learning (OD-ICL), this distribution change happens in the input space without affecting the label distribution. In order to adapt to such changes, the model being trained risks forgetting previously learned knowledge (stability). On the other hand, enforcing that the model preserves past knowledge will cause it to fail to learn new concepts (plasticity). We propose Online Domain Incremental Networks (ODIN), a novel method to alleviate catastrophic forgetting by automatically detecting the end of a task using concept drift detection. As a consequence, ODIN does not require the specification of task ids. ODIN maintains a pool of NNs, each trained on a single task and frozen for further updates. A Task Predictor (TP) is trained to select the most suitable NN from the frozen pool for prediction. We compare ODIN against popular regularization and replay methods. It outperforms regularization methods and achieves comparable predictive performance to replay methods.
Fichier non déposé

Dates et versions

hal-04468398 , version 1 (20-02-2024)

Identifiants

Citer

Nuwan Gunasekara, Heitor Murilo Gomes, Albert Bifet, Bernhard Pfahringer. Adaptive Neural Networks for Online Domain Incremental Continual Learning. Discovery Science - 25th International Conference, DS 2022, Montpellier, France, October 10-12, 2022, Proceedings, Oct 2022, Montpellier, France. pp.89--103, ⟨10.1007/978-3-031-18840-4_7⟩. ⟨hal-04468398⟩
3 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More