Prediction of perceptual similarity based on time-domain models of auditory perception
Prédiction de la similarité perceptive basée sur des modèles temporels de perception auditive
Résumé
Objects or situations in an everyday context are unlikely to be experienced twice in the same way. The more exposed an individual is to a
given object or situation, the more familiar he or she becomes with that object or situation. While listening to a sound object, we may find that
it resembles another sound with which we are familiar. In this case we may label both sounds as being “similar”. Similarity assessments may
indicate whether two or more sound stimuli share common perceptual properties. Let us consider a sound quality evaluation between the ref-
erence sound A and the test sound B. The test sound B can be chosen as being (1) a modified version of A, (2) a synthesised version of A, or
(3) a sound that is believed to be similar to A. An evaluation of the first type (1) is useful to study which properties of sound A are perceptually
prominent. An evaluation of the second type (2) can be used to validate a computational model that accounts for the theory that is believed to
be relevant to recreate sound A. An evaluation of the third type (3) can lead to a measure of perceptual distance between sounds A and B. The
work in this dissertation is mainly concerned with this latter type of evaluation.
The goal of this research work was to gain insights into human performance in a similarity task. For this purpose, the similarity of a set of
sounds was first experimentally assessed. Subsequently, the same experimental framework was implemented and used as input to a state-of-the-
art model of auditory perception. The hypothesis was that the similarity assessments obtained from the auditory model are significantly correlated
with those obtained experimentally.
In this study we chose to compare sounds using the internal (sound) representations delivered by an auditory model. The model, referred
to as perception model (PEMO), offers a unified framework that has been successfully used to simulate a number of auditory phenomena
such as masking and modulation tasks. The advantage of using a unified framework is implicitly emphasised in Chapter 2, where recorded
and synthesised sounds of an instrument called Hummer are compared (type 2 task) using three auditory models that deliver four psychoacoustic descriptors: Loudness, loudness fluctuations, fluctuation strength, and roughness. The model estimates are compared using the concept
of just-noticeable difference (JND), with one JND value for each of the four psychoacoustic descriptors. If the descriptors differ by less than one
JND, the sounds are considered to be perceptually identical along the evaluated dimensions.
In Chapter 3 a new method to assess the perceptual similarity between sounds is introduced and validated. In the so-called instrument-in-noise
method two sounds are compared using a three-alternative forced-choice paradigm (3-AFC). The reference sound is presented twice and the test
sound is presented once. The task of the participant is to identify in which of the three sound intervals the test sound was played. One of
the key aspects of this method is that a background noise is added to manipulate the difficulty of the task. This allows to assess the similarity
between two sounds as a performance task. The background noise needs to have similar spectro-temporal properties to those of the test sounds.
For this purpose a noise generation algorithm similar to the ICRA noises was adopted. Two sounds that are similar tolerate a low background
(ICRA) noise to correctly discriminate one from the other in contrast to the case of two sounds that are more dissimilar, where more (ICRA)
noise needs to be added before the participant’s performance decreases. The sound stimuli consisted of recordings of a single note from seven historical pianos. With seven sound stimuli, 21 possible piano pairs can be evaluated. Twenty participants were asked to compare those 21 piano
pairs using two methods: (1) the instrument-in-noise method, and (2) the method of triadic comparisons. The discrimination thresholds from
the instrument-in-noise method were significantly correlated with the similarity assessment obtained from the method of triadic comparisons.
In Chapter 4 the participant’s performance for the instrument-in-noise test is simulated using the same piano sounds and experimental paradigm
as in Chapter 3 but using an “artificial listener”. The artificial listener uses internal representations obtained with the PEMO model and decides whether two representations are distinct enough to be judged as “different”. This decision is based on the concept of optimal detector taken from signal detection theory. Both, the peripheral stages (that deliver the internal representations) as well as the central stage (the artificial listener) of the PEMO model are described in detail in this chapter. The discrimination thresholds obtained with the PEMO model are significantly correlated with the experimental thresholds.
In Chapter 5, the same seven piano sounds of Chapters 3 and 4 but considering a reverberant environment (early decay time of 3.0 s) were
perceptually evaluated. Discrimination thresholds obtained from twenty new participants were assessed and subsequently simulated using the
PEMO model. The results had a similar (significant) correlation between experimental and simulated thresholds, as observed when comparing the
results of Chapters 3 and 4.
In Chapter 6 a binaural model that has the same peripheral stages as the PEMO model, but using a different central processor, is used to simulate the perceived reverberation (reverberance) of orchestra sounds in eight different acoustic environments. The main goal of this chapter is to show one example of application that further extends the use of the auditory models. The reverberance estimates obtained from the binaural model were compared with the experimental results of a multi-stimulus comparison task. The experiment considered 8 instruments and they were evaluated by 24 participants. The multi-stimulus comparison is an alternative and faster way to compare sounds pairwise and it can be used to develop perceptual scales. The experimental reverberance estimates were significantly correlated with the simulated reverberance estimates.
The work presented in this dissertation supports the use of a unified auditory modelling framework to simulate a perceptual similarity task using sounds that are non-artificial. The unified framework was used to evaluate two similar sets of sounds: single-note recordings from seven piano sounds without (Chapters 3 and 4) and with reverberation (Chapter 5). The experimental paradigm, that we named instrument-in-noise test, can be further used to evaluate other musical instruments as far as the sounds to be evaluated have the same duration and are tuned to the same frequency. These aspects are relevant to appropriately generate noises that match the spectro-temporal properties of the sounds being tested.
Objects or situations in an everyday context are unlikely to be experienced twice in the same way. The more exposed an individual is to a
given object or situation, the more familiar he or she becomes with that object or situation. While listening to a sound object, we may find that
it resembles another sound with which we are familiar. In this case we may label both sounds as being “similar”. Similarity assessments may
indicate whether two or more sound stimuli share common perceptual properties. Let us consider a sound quality evaluation between the ref-
erence sound A and the test sound B. The test sound B can be chosen as being (1) a modified version of A, (2) a synthesised version of A, or
(3) a sound that is believed to be similar to A. An evaluation of the first type (1) is useful to study which properties of sound A are perceptually
prominent. An evaluation of the second type (2) can be used to validate a computational model that accounts for the theory that is believed to
be relevant to recreate sound A. An evaluation of the third type (3) can lead to a measure of perceptual distance between sounds A and B. The
work in this dissertation is mainly concerned with this latter type of evaluation.
The goal of this research work was to gain insights into human performance in a similarity task. For this purpose, the similarity of a set of
sounds was first experimentally assessed. Subsequently, the same experimental framework was implemented and used as input to a state-of-the-
art model of auditory perception. The hypothesis was that the similarity assessments obtained from the auditory model are significantly correlated
with those obtained experimentally.
In this study we chose to compare sounds using the internal (sound) representations delivered by an auditory model. The model, referred
to as perception model (PEMO), offers a unified framework that has been successfully used to simulate a number of auditory phenomena
such as masking and modulation tasks. The advantage of using a unified framework is implicitly emphasised in Chapter 2, where recorded
and synthesised sounds of an instrument called Hummer are compared (type 2 task) using three auditory models that deliver four psychoacoustic descriptors: Loudness, loudness fluctuations, fluctuation strength, and roughness. The model estimates are compared using the concept
of just-noticeable difference (JND), with one JND value for each of the four psychoacoustic descriptors. If the descriptors differ by less than one
JND, the sounds are considered to be perceptually identical along the evaluated dimensions.
In Chapter 3 a new method to assess the perceptual similarity between sounds is introduced and validated. In the so-called instrument-in-noise
method two sounds are compared using a three-alternative forced-choice paradigm (3-AFC). The reference sound is presented twice and the test
sound is presented once. The task of the participant is to identify in which of the three sound intervals the test sound was played. One of
the key aspects of this method is that a background noise is added to manipulate the difficulty of the task. This allows to assess the similarity
between two sounds as a performance task. The background noise needs to have similar spectro-temporal properties to those of the test sounds.
For this purpose a noise generation algorithm similar to the ICRA noises was adopted. Two sounds that are similar tolerate a low background
(ICRA) noise to correctly discriminate one from the other in contrast to the case of two sounds that are more dissimilar, where more (ICRA)
noise needs to be added before the participant’s performance decreases. The sound stimuli consisted of recordings of a single note from seven historical pianos. With seven sound stimuli, 21 possible piano pairs can be evaluated. Twenty participants were asked to compare those 21 piano
pairs using two methods: (1) the instrument-in-noise method, and (2) the method of triadic comparisons. The discrimination thresholds from
the instrument-in-noise method were significantly correlated with the similarity assessment obtained from the method of triadic comparisons.
In Chapter 4 the participant’s performance for the instrument-in-noise test is simulated using the same piano sounds and experimental paradigm
as in Chapter 3 but using an “artificial listener”. The artificial listener uses internal representations obtained with the PEMO model and decides whether two representations are distinct enough to be judged as “different”. This decision is based on the concept of optimal detector taken from signal detection theory. Both, the peripheral stages (that deliver the internal representations) as well as the central stage (the artificial listener) of the PEMO model are described in detail in this chapter. The discrimination thresholds obtained with the PEMO model are significantly correlated with the experimental thresholds.
In Chapter 5, the same seven piano sounds of Chapters 3 and 4 but considering a reverberant environment (early decay time of 3.0 s) were
perceptually evaluated. Discrimination thresholds obtained from twenty new participants were assessed and subsequently simulated using the
PEMO model. The results had a similar (significant) correlation between experimental and simulated thresholds, as observed when comparing the
results of Chapters 3 and 4.
In Chapter 6 a binaural model that has the same peripheral stages as the PEMO model, but using a different central processor, is used to simulate the perceived reverberation (reverberance) of orchestra sounds in eight different acoustic environments. The main goal of this chapter is to show one example of application that further extends the use of the auditory models. The reverberance estimates obtained from the binaural model were compared with the experimental results of a multi-stimulus comparison task. The experiment considered 8 instruments and they were evaluated by 24 participants. The multi-stimulus comparison is an alternative and faster way to compare sounds pairwise and it can be used to develop perceptual scales. The experimental reverberance estimates were significantly correlated with the simulated reverberance estimates.
The work presented in this dissertation supports the use of a unified auditory modelling framework to simulate a perceptual similarity task using sounds that are non-artificial. The unified framework was used to evaluate two similar sets of sounds: single-note recordings from seven piano sounds without (Chapters 3 and 4) and with reverberation (Chapter 5). The experimental paradigm, that we named instrument-in-noise test, can be further used to evaluate other musical instruments as far as the sounds to be evaluated have the same duration and are tuned to the same frequency. These aspects are relevant to appropriately generate noises that match the spectro-temporal properties of the sounds being tested.
Fichier principal
Osses2018-TUe-20180820-PhD-thesis-ver-2+cover.pdf (4.79 Mo)
Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)