Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue EURASIP Journal on Audio, Speech, and Music Processing Année : 2021

Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques

Résumé

Instrumental playing techniques such as vibratos, glissandos, and trills often denote musical expressivity, both in classical and folk contexts. However, most existing approaches to music similarity retrieval fail to describe timbre beyond the so-called "ordinary" technique, use instrument identity as a proxy for timbre quality, and do not allow for customization to the perceptual idiosyncrasies of a new subject. In this article, we ask 31 human participants to organize 78 isolated notes into a set of timbre clusters. Analyzing their responses suggests that timbre perception operates within a more flexible taxonomy than those provided by instruments or playing techniques alone. In addition, we propose a machine listening model to recover the cluster graph of auditory similarities across instruments, mutes, and techniques. Our model relies on joint time-frequency scattering features to extract spectrotemporal modulations as acoustic features. Furthermore, it minimizes triplet loss in the cluster graph by means of the large-margin nearest neighbor (LMNN) metric learning algorithm. Over a dataset of 9346 isolated notes, we report a state-of-the-art average precision at rank five (AP@5) of 99.0% ± 1. An ablation study demonstrates that removing either the joint time-frequency scattering transform or the metric learning algorithm noticeably degrades performance.
Fichier principal
Vignette du fichier
paper.pdf (1.93 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03000693 , version 1 (12-11-2020)

Identifiants

  • HAL Id : hal-03000693 , version 1

Citer

Vincent Lostanlen, Christian El-Hajj, Mathias Rossignol, Grégoire Lafay, Joakim Andén, et al.. Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques. EURASIP Journal on Audio, Speech, and Music Processing, 2021, 1 (1), pp.1-21. ⟨hal-03000693⟩
74 Consultations
79 Téléchargements

Partager

Gmail Facebook X LinkedIn More