Comparison of subjective methodologies for local perception of distortion in videos and impact on objective metrics resolving power - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Comparison of subjective methodologies for local perception of distortion in videos and impact on objective metrics resolving power

Résumé

Different subjective methodologies exist to collect data on human perception of distortions, from rating methodologies with single or double stimuli to ranking with pairwise comparisons. The Maximum Likelihood Difference Scaling (MLDS) method uses triplet/quadruplet-based comparisons as a ranking task. Participants compare intervals inside pairs of stimuli: (a,b) and (c,d). The task is to rank if they perceive greater differences between (a,b) or (c,d). From these comparisons’ judgments, we can place the assessed stimuli on a perceptual scale (e.g., from low to high quality) with the help of a mathematical solver. However, one limitation is that the perceptual scales retrieved from stimuli of multiple contents are usually different. We previously offered a solution to measure the inter-content scale of multiple contents. In this work, we compare multiple rating and ranking methodologies. We examine obtained subjective quality scores regarding precision by analyzing discriminability in the scores, efficiency by comparing fixed experimental effort costs, and robustness of retrieve estimates to outliers and spammer behaviors. In this work, we put data quality, experimental cost, and resolving power into relation. We show how discriminability in the data impacts the resolving power of popular objective quality metrics. Our findings are that higher-performing metrics require higher-quality data to reveal their full potential.
Fichier principal
Vignette du fichier
HVEI_2024_methods_comparison_PU.pdf (2.47 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04458332 , version 1 (18-02-2024)

Identifiants

  • HAL Id : hal-04458332 , version 1

Citer

Andreas Pastor, Lukáš Krasula, Xiaoqing Zhu, Zhi Li, Patrick Le Callet. Comparison of subjective methodologies for local perception of distortion in videos and impact on objective metrics resolving power. 2024. ⟨hal-04458332⟩
18 Consultations
23 Téléchargements

Partager

Gmail Facebook X LinkedIn More