"Discriminability-Experimental Cost" tradeoff in subjective video quality assessment of codec: DCR with EVP rating scale versus ACR-HR - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

"Discriminability-Experimental Cost" tradeoff in subjective video quality assessment of codec: DCR with EVP rating scale versus ACR-HR

Résumé

This work compares two subjective studies conducted in a controlled laboratory environment on SDR HD, UHD, and HDR UHD contents using naive observers. The goal of these tests is to compare the precision and accuracy of a modified Degradation Category Rating (DCR) and Absolute Category Rating with Hidden Reference (ACR-HR) subjective methods for video quality assessment. The modified version of the DCR method includes a repetition of both reference and distorted stimuli; and utilizes an 11-grade rating scale from Expert Viewing Protocol (EVP) of ITU-R BT.500-15 standards. In the second subjective protocol, ACR-HR operates without repetition and with the 5-grade quality scale from ITU standards. We perform an extensive analysis of the scale usage and compare Mean Opinion Scores (MOS) scores discriminability in both subjective studies. We show that both methods can retrieve accurate MOS. However, the ACR-HR method achieves better discriminability among MOS than DCR with EVP rating scale, while reducing by a factor of two the experimental effort, i.e., the cost of the experiment. The findings of this work give new insight into how to perform cost-efficient subjective tests for video quality estimation with naive observers, and how to retrieve good MOS estimates.
Fichier principal
Vignette du fichier
PCS2024_ACR-HR_DCR-EVP.pdf (1.48 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04363990 , version 1 (26-12-2023)

Identifiants

  • HAL Id : hal-04363990 , version 1

Citer

Andréas Pastor, Pierre David, Ioannis Katsavounidis, Lukas Krasula, Hassene Tmar, et al.. "Discriminability-Experimental Cost" tradeoff in subjective video quality assessment of codec: DCR with EVP rating scale versus ACR-HR. 2023. ⟨hal-04363990⟩
120 Consultations
133 Téléchargements

Partager

Gmail Mastodon Facebook X LinkedIn More