Exploring Crowdsourcing for Subjective Quality Assessment of 3D Graphics - Archive ouverte HAL
Communication Dans Un Congrès Année : 2021

Exploring Crowdsourcing for Subjective Quality Assessment of 3D Graphics

Yana Nehmé
  • Fonction : Auteur
  • PersonId : 177524
  • IdHAL : yana-nehme
Florent Dupont
Jean-Philippe Farrugia
Guillaume Lavoué

Résumé

Multimedia subjective quality assessment experiments are the most prominent and reliable way to evaluate the visual quality as perceived by human observers. Along with laboratory (lab) subjective experiments, crowdsourcing (CS) experiments have become very popular in recent years, e.g., during the COVID-19 pandemic these experiments provide an alternative to lab tests. However, conducting subjective quality assessment tests in CS raises many challenges: internet connection quality, lack of control on participants' environment, participants' consistency and reliability, etc. In this work, we evaluate the performance of CS studies for 3D graphics quality assessment. To this end, we conducted a CS experiment based on the double stimulus impairment scale method and using a dataset of 80 meshes with diffuse color information corrupted by various distortions. We compared its results with those previously obtained in a lab study conducted on the same dataset and in a virtual reality environment. Results show that under controlled conditions and with appropriate participant screening strategies, a CS experiment can be as accurate as a lab experiment.
Fichier principal
Vignette du fichier
MMSP_2021.pdf (997.74 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03556845 , version 1 (04-02-2022)

Identifiants

Citer

Yana Nehmé, Patrick Le Callet, Florent Dupont, Jean-Philippe Farrugia, Guillaume Lavoué. Exploring Crowdsourcing for Subjective Quality Assessment of 3D Graphics. IEEE International Workshop on Multimedia Signal Processing (MMSP), Oct 2021, Tampere, Ireland. ⟨10.1109/MMSP53017.2021.9733634⟩. ⟨hal-03556845⟩
97 Consultations
187 Téléchargements

Altmetric

Partager

More