DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue AES - Journal of the Audio Engineering Society Audio-Accoustics-Application Année : 2023

DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge

Résumé

Instrument sound synthesis using deep neural networks has received numerous improvements over the last couple of years. Among them, the Differentiable Digital Signal Processing (DDSP) framework has modernized the spectral modeling paradigm by including signal-based synthesizers and effects into fully differentiable architectures. The present work extends the applications of DDSP to the task of polyphonic sound synthesis, with the proposal of a differentiable piano synthesizer conditioned on MIDI inputs. The model architecture is motivated by high-level acoustic modeling knowledge of the instrument, which, along with the sound structure priors inherent to the DDSP components, makes for a lightweight, interpretable, and realistic-sounding piano model. A subjective listening test has revealed that the proposed approach achieves better sound quality than a state-of-the-art neural-based piano synthesizer, but physical-modeling-based models still hold the best quality. Leveraging its interpretability and modularity, a qualitative analysis of the model behavior was also conducted: it highlights where additional modeling knowledge and optimization procedures could be inserted in order to improve the synthesis quality and the manipulation of sound properties. Eventually, the proposed differentiable synthesizer can be further used with other deep learning models for alternative musical tasks handling polyphonic audio and symbolic data.
Fichier principal
Vignette du fichier
DDSP_Piano_JAES_final.pdf (5.49 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04073770 , version 1 (19-04-2023)
hal-04073770 , version 2 (15-09-2023)

Identifiants

Citer

Lenny Renault, Rémi Mignot, Axel Roebel. DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge. AES - Journal of the Audio Engineering Society Audio-Accoustics-Application, 2023, Special Issue on New Trends in Audio Effects, Part 2, 71 (9), pp.552-565. ⟨10.17743/jaes.2022.0102⟩. ⟨hal-04073770v2⟩
259 Consultations
186 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More