Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case

Résumé

In order to foster inclusive education, automatic systems that can adapt textbooks to make them accessible to children with Developmental Coordination Disorder (DCD) are necessary. In this context, we propose a task to classify exercises according to their DCD adaptation type. We introduce a challenging exercise dataset extracted from French textbooks, with two major difficulties: limited and unbalanced, noisy data. To set a baseline on the dataset, we use state-of-the-art models combined through early and late fusion techniques to take advantage of text and vision/layout modalities. Our approach achieves an overall accuracy of 0.802. However, the experiments show the difficulty of the task, especially for minority classes, where the accuracy drops to 0.583.
Fichier principal
Vignette du fichier
Lincker_Noisy-and-Unbalanced-Multimodal-Document-Classification.pdf (850.58 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY - Paternité

Dates et versions

hal-04221023 , version 1 (28-09-2023)

Licence

Paternité

Identifiants

Citer

Élise Lincker, Camille Guinaudeau, Olivier Pons, Jérôme Dupire, Céline Hudelot, et al.. Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case. 20th International Conference on Content-based Multimedia Indexing (CBMI 2023), Sep 2023, Orléans, France. ⟨10.1145/3617233.3617239⟩. ⟨hal-04221023⟩
114 Consultations
98 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More