Don't forget, there is more than forgetting: new metrics for Continual Learning - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Don't forget, there is more than forgetting: new metrics for Continual Learning

Résumé

Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and forward knowledge transfer, memory overhead as well as computational efficiency. Drawing inspiration from the standard Multi-Attribute Value Theory (MAVT) we further propose to fuse these metrics into a single score for ranking purposes and we evaluate our proposal with five continual learning strategies on the iCIFAR-100 continual learning benchmark.
Fichier principal
Vignette du fichier
Don_t_forget__there_is_more_than_forgetting NIPS18CL_WS.pdf (461.69 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01951488 , version 1 (11-12-2018)

Identifiants

  • HAL Id : hal-01951488 , version 1

Citer

Natalia Díaz-Rodríguez, Vincenzo Lomonaco, David Filliat, Davide Maltoni. Don't forget, there is more than forgetting: new metrics for Continual Learning. Workshop on Continual Learning, NeurIPS 2018 (Neural Information Processing Systems, Dec 2018, Montreal, Canada. ⟨hal-01951488⟩
117 Consultations
1022 Téléchargements

Partager

Gmail Facebook X LinkedIn More