Evaluating the effects of task design on unfamiliar Francophone listener and automatic speaker identification performance - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Multimedia Tools and Applications Année : 2023

Evaluating the effects of task design on unfamiliar Francophone listener and automatic speaker identification performance

Natalia Tomashenko
Alain Ghio
Jean-François Bonastre

Résumé

Many questions remain with regards to how context affects perceptual and automatic speaker identification performance. To examine the effects of task design on perceptual speaker identification performance, three tasks were developed, including lineup and binary tasks, as well as a novel clustering task. Speech recordings of native French speakers were compared similarly across tasks evaluated by unfamiliar Francophone listeners. True positive (sensitivity) and true negative (specificity) response rates across tasks were measured. Our results showed participants had similar sensitivity and specificity for binary (88%) and clustering (84%) tasks, but random selection rates for the lineup task. Pearson correlation procedures were used to evaluate the efficiency of scores produced by a state-of-the-art automatic speaker verification to model perceptual responses (equal error rate = 89%). Automatic scores modelled lineup (r 2 = 0.6) and clustering (r 2 = 0.5) task accuracy quite well, however, they were less robust when modelling binary task responses (r 2 =-0.2). The results underscore the role task design plays in shaping perceptual responses, which, in turn, affects the modelling effectiveness of automatic scores. As evidence points to humans and algorithms modelling speakers differently, our findings suggest automatic speaker identification performance might be improved with a greater understanding on how context shapes perceptual responses.

Domaines

Linguistique
Fichier principal
Vignette du fichier
2d052e71-f880-41c9-aaf2-f20bd7d3bf5b.pdf (1.21 Mo) Télécharger le fichier
11042_2023_15391_OnlinePDF.pdf (1.44 Mo) Télécharger le fichier
Origine : Publication financée par une institution

Dates et versions

hal-04138760 , version 1 (23-06-2023)

Identifiants

Citer

Benjamin O'Brien, Christine Meunier, Natalia Tomashenko, Alain Ghio, Jean-François Bonastre. Evaluating the effects of task design on unfamiliar Francophone listener and automatic speaker identification performance. Multimedia Tools and Applications, 2023, ⟨10.1007/s11042-023-15391-0⟩. ⟨hal-04138760⟩
33 Consultations
48 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More