PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective

Résumé

In this paper, we address the problem of pitch estimation using Self Supervised Learning (SSL). The SSL paradigm we use is equivariance to pitch transposition, which enables our model to accurately perform pitch estimation on monophonic audio after being trained only on a small unlabeled dataset. We use a lightweight ($<$ 30k parameters) Siamese neural network that takes as inputs two different pitch-shifted versions of the same audio represented by its Constant-Q Transform. To prevent the model from collapsing in an encoder-only setting, we propose a novel class-based transposition-equivariant objective which captures pitch information. Furthermore, we design the architecture of our network to be transposition-preserving by introducing learnable Toeplitz matrices. We evaluate our model for the two tasks of singing voice and musical instrument pitch estimation and show that our model is able to generalize across tasks and datasets while being lightweight, hence remaining compatible with low-resource devices and suitable for real-time applications. In particular, our results surpass self-supervised baselines and narrow the performance gap between self-supervised and supervised methods for pitch estimation.
Fichier principal
Vignette du fichier
PESTO.pdf (761.09 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY - Paternité

Dates et versions

hal-04260042 , version 1 (26-10-2023)

Licence

Paternité

Identifiants

Citer

Alain Riou, Stefan Lattner, Gaëtan Hadjeres, Geoffroy Peeters. PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective. International Society for Music Information Retrieval Conference (ISMIR 2023), Nov 2023, Milan, Italy. ⟨10.48550/arXiv.2309.02265⟩. ⟨hal-04260042⟩
216 Consultations
230 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More