PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective - Archive ouverte HAL Access content directly
Conference Papers Year : 2023

PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective


In this paper, we address the problem of pitch estimation using Self Supervised Learning (SSL). The SSL paradigm we use is equivariance to pitch transposition, which enables our model to accurately perform pitch estimation on monophonic audio after being trained only on a small unlabeled dataset. We use a lightweight ($<$ 30k parameters) Siamese neural network that takes as inputs two different pitch-shifted versions of the same audio represented by its Constant-Q Transform. To prevent the model from collapsing in an encoder-only setting, we propose a novel class-based transposition-equivariant objective which captures pitch information. Furthermore, we design the architecture of our network to be transposition-preserving by introducing learnable Toeplitz matrices. We evaluate our model for the two tasks of singing voice and musical instrument pitch estimation and show that our model is able to generalize across tasks and datasets while being lightweight, hence remaining compatible with low-resource devices and suitable for real-time applications. In particular, our results surpass self-supervised baselines and narrow the performance gap between self-supervised and supervised methods for pitch estimation.
Fichier principal
Vignette du fichier
PESTO.pdf (761.09 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Licence : CC BY - Attribution

Dates and versions

hal-04260042 , version 1 (26-10-2023)





Alain Riou, Stefan Lattner, Gaëtan Hadjeres, Geoffroy Peeters. PESTO: Pitch Estimation with Self-supervised Transposition-equivariant Objective. International Society for Music Information Retrieval Conference (ISMIR 2023), Nov 2023, Milan, Italy. ⟨10.48550/arXiv.2309.02265⟩. ⟨hal-04260042⟩
154 View
149 Download



Gmail Facebook X LinkedIn More