DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge - Archive ouverte HAL Access content directly
Journal Articles AES - Journal of the Audio Engineering Society Audio-Accoustics-Application Year : 2023

DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge


Instrument sound synthesis using deep neural networks has received numerous improvements over the last couple of years. Among them, the Differentiable Digital Signal Processing (DDSP) framework has modernized the spectral modeling paradigm by including signal-based synthesizers and effects into fully differentiable architectures. The present work extends the applications of DDSP to the task of polyphonic sound synthesis, with the proposal of a differentiable piano synthesizer conditioned on MIDI inputs. The model architecture is motivated by high-level acoustic modeling knowledge of the instrument, which, along with the sound structure priors inherent to the DDSP components, makes for a lightweight, interpretable, and realistic-sounding piano model. A subjective listening test has revealed that the proposed approach achieves better sound quality than a state-of-the-art neural-based piano synthesizer, but physical-modeling-based models still hold the best quality. Leveraging its interpretability and modularity, a qualitative analysis of the model behavior was also conducted: it highlights where additional modeling knowledge and optimization procedures could be inserted in order to improve the synthesis quality and the manipulation of sound properties. Eventually, the proposed differentiable synthesizer can be further used with other deep learning models for alternative musical tasks handling polyphonic audio and symbolic data.
Fichier principal
Vignette du fichier
DDSP_Piano_JAES_final.pdf (5.49 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04073770 , version 1 (19-04-2023)
hal-04073770 , version 2 (15-09-2023)



Lenny Renault, Rémi Mignot, Axel Roebel. DDSP-Piano: a Neural Sound Synthesizer Informed by Instrument Knowledge. AES - Journal of the Audio Engineering Society Audio-Accoustics-Application, 2023, Special Issue on New Trends in Audio Effects, Part 2, 71 (9), pp.552-565. ⟨10.17743/jaes.2022.0102⟩. ⟨hal-04073770v2⟩
231 View
139 Download



Gmail Facebook X LinkedIn More