Evaluating the effects of task design on unfamiliar Francophone listener and automatic speaker identification performance - Archive ouverte HAL
Journal Articles Multimedia Tools and Applications Year : 2023

Evaluating the effects of task design on unfamiliar Francophone listener and automatic speaker identification performance

Natalia Tomashenko
Alain Ghio
Jean-François Bonastre

Abstract

Many questions remain with regards to how context affects perceptual and automatic speaker identification performance. To examine the effects of task design on perceptual speaker identification performance, three tasks were developed, including lineup and binary tasks, as well as a novel clustering task. Speech recordings of native French speakers were compared similarly across tasks evaluated by unfamiliar Francophone listeners. True positive (sensitivity) and true negative (specificity) response rates across tasks were measured. Our results showed participants had similar sensitivity and specificity for binary (88%) and clustering (84%) tasks, but random selection rates for the lineup task. Pearson correlation procedures were used to evaluate the efficiency of scores produced by a state-of-the-art automatic speaker verification to model perceptual responses (equal error rate = 89%). Automatic scores modelled lineup (r 2 = 0.6) and clustering (r 2 = 0.5) task accuracy quite well, however, they were less robust when modelling binary task responses (r 2 =-0.2). The results underscore the role task design plays in shaping perceptual responses, which, in turn, affects the modelling effectiveness of automatic scores. As evidence points to humans and algorithms modelling speakers differently, our findings suggest automatic speaker identification performance might be improved with a greater understanding on how context shapes perceptual responses.

Domains

Linguistics
Fichier principal
Vignette du fichier
2d052e71-f880-41c9-aaf2-f20bd7d3bf5b.pdf (1.21 Mo) Télécharger le fichier
11042_2023_15391_OnlinePDF.pdf (1.44 Mo) Télécharger le fichier
Origin Publication funded by an institution

Dates and versions

hal-04138760 , version 1 (23-06-2023)

Identifiers

Cite

Benjamin O'Brien, Christine Meunier, Natalia Tomashenko, Alain Ghio, Jean-François Bonastre. Evaluating the effects of task design on unfamiliar Francophone listener and automatic speaker identification performance. Multimedia Tools and Applications, 2023, ⟨10.1007/s11042-023-15391-0⟩. ⟨hal-04138760⟩
51 View
96 Download

Altmetric

Share

More