GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph

Noa Garcia
  • Fonction : Auteur
  • PersonId : 1098949
Chenhui Chu
  • Fonction : Auteur
  • PersonId : 1098950
Yuta Nakashima
  • Fonction : Auteur
  • PersonId : 1098951
Hajime Nagahara
  • Fonction : Auteur
  • PersonId : 1098952

Résumé

The rise of digitization of cultural documents offers large-scale contents, opening the road for development of AI systems in order to preserve, search, and deliver cultural heritage. To organize such cultural content also means to classify them, a task that is very familiar to modern computer science. Contextual information is often the key to structure such real world data, and we propose to use it in form of a knowledge graph. Such a knowledge graph, combined with content analysis, enhances the notion of proximity between artworks so it improves the performances in classification tasks. In this paper, we propose a novel use of a knowledge graph, that is constructed on annotated data and pseudo-labeled data. With label propagation, we boost artwork classification by training a model using a graph convolutional network, relying on the relationships between entities of the knowledge graph. Following a transductive learning framework, our experiments show that relying on a knowledge graph modeling the relations between labeled data and unlabeled data allows to achieve state-of-the-art results on multiple classification tasks on a dataset of paintings, and on a dataset of Buddha statues. Additionally, we show state-of-the-art results for the difficult case of dealing with unbalanced data, with the limitation of disregarding classes with extremely low degrees in the knowledge graph.
Fichier principal
Vignette du fichier
GCNBoost-hal.pdf (5.31 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03228787 , version 1 (18-05-2021)

Identifiants

  • HAL Id : hal-03228787 , version 1

Citer

Cheikh Brahim El Vaigh, Noa Garcia, Benjamin Renoust, Chenhui Chu, Yuta Nakashima, et al.. GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph. ICMR 2021 - ACM International Conference on Multimedia Retrieval, Aug 2021, Taipei, Taiwan. ⟨hal-03228787⟩
108 Consultations
158 Téléchargements

Partager

Gmail Facebook X LinkedIn More