A soft nearest-neighbor framework for continual semi-supervised learning - Archive ouverte HAL Access content directly
Conference Papers Year : 2023

A soft nearest-neighbor framework for continual semi-supervised learning

Abstract

Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-supervised learning--a setting where not all the data samples are labeled. A primary issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled samples. We leverage the power of nearest-neighbor classifiers to nonlinearly partition the feature space and flexibly model the underlying data distribution thanks to its non-parametric nature. This enables the model to learn a strong representation for the current task, and distill relevant information from previous tasks. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a solid state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR-100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations). Finally, our method works well on both low and high resolution images and scales seamlessly to more complex datasets such as ImageNet-100. The code is publicly available on https://github.com/kangzhiq/NNCSL.
Fichier principal
Vignette du fichier
sgfdzhfsvsncpvkkcmyxmtrhygzhnnxq.pdf (882.77 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03893056 , version 1 (10-12-2022)
hal-03893056 , version 2 (05-04-2023)
hal-03893056 , version 3 (11-09-2023)

Licence

Identifiers

  • HAL Id : hal-03893056 , version 3

Cite

Zhiqi Kang, Enrico Fini, Moin Nabi, Elisa Ricci, Karteek Alahari. A soft nearest-neighbor framework for continual semi-supervised learning. ICCV 2023 - IEEE/CVF International Conference on Computer Vision, Oct 2023, Paris, France. ⟨hal-03893056v3⟩
146 View
68 Download

Share

Gmail Mastodon Facebook X LinkedIn More