Towards Inclusive Education: Multimodal Classification of Textbook Images for Accessibility - Archive ouverte HAL
Communication Dans Un Congrès Année : 2025

Towards Inclusive Education: Multimodal Classification of Textbook Images for Accessibility

Résumé

To foster inclusive education, accessible educational materials must be tailored to meet the diverse needs of students, especially students with disabilities. Images in educational materials play an important role in understanding textbook exercises but can also distract such students and make them more challenging. Therefore, we need an adaptive system that identifies non-essential images to reduce cognitive load and distractions for students with disabilities. Accordingly, this work proposes a computational framework to categorize textbook exercise images, facilitating their inclusion in accessible textbooks and enhancing learning for visually impaired and neurodevelopmental disorders students. Using three French textbook exercise datasets of 652 (text, image) pairs, we compared monomodal (text-only) and multimodal (text and image) classification approaches. We found that text-based models, particularly CamemBERT, excel in classifying images, achieving an accuracy rate of 85.25% on French text data. We also use Local Interpretable Modelagnostic Explanation for model interpretability and conduct qualitative analyses to deepen insights into model performance. This work is only a very first step towards an automatic translation of inclusive textbooks. Moreover, this paper revealed that this first step is already very challenging. We hope to draw more researchers' attention to this problem.
Fichier principal
Vignette du fichier
Yadav2025_Towards_inclusive_education.pdf (7.82 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04860245 , version 1 (31-12-2024)

Identifiants

  • HAL Id : hal-04860245 , version 1

Citer

Saumya Yadav, Elise Lincker, Caroline Huron, Stéphanie Martin, Camille Guinaudeau, et al.. Towards Inclusive Education: Multimodal Classification of Textbook Images for Accessibility. Multimedia Modelling 2025, Jan 2025, Nara, Japan. ⟨hal-04860245⟩
0 Consultations
0 Téléchargements

Partager

More