Learning to Solve Inverse Problems for Perceptual Sound Matching - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue IEEE/ACM Transactions on Audio, Speech and Language Processing Année : 2024

Learning to Solve Inverse Problems for Perceptual Sound Matching

Résumé

Perceptual sound matching (PSM) aims to find the input parameters to a synthesizer so as to best imitate an audio target. Deep learning for PSM optimizes a neural network to analyze and reconstruct prerecorded samples. In this context, our article addresses the problem of designing a suitable loss function when the training set is generated by a differentiable synthesizer. Our main contribution is perceptual--neural--physical loss (PNP), which aims at addressing a tradeoff between perceptual relevance and computational efficiency. The key idea behind PNP is to linearize the effect of synthesis parameters upon auditory features in the vicinity of each training sample. The linearization procedure is massively parallelizable, can be precomputed, and offers a 100-fold speedup during gradient descent compared to differentiable digital signal processing (DDSP). We show that PNP is able to accelerate DDSP with joint time--frequency scattering transform (JTFS) as auditory feature while preserving its perceptual fidelity. Additionally, we evaluate the impact of other design choices in PSM: parameter rescaling, pretraining, auditory representation, and gradient clipping. We report state-of-the-art results on both datasets and find that PNP-accelerated JTFS has greater influence on PSM performance than any other design choice.

Domaines

Son [cs.SD]
Fichier principal
Vignette du fichier
han2024taslp.pdf (679.6 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04580765 , version 1 (20-05-2024)

Licence

Identifiants

Citer

Han Han, Vincent Lostanlen, Mathieu Lagrange. Learning to Solve Inverse Problems for Perceptual Sound Matching. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2024, 32, pp.2605-2615. ⟨10.1109/TASLP.2024.3393738⟩. ⟨hal-04580765⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More