Mesostructures: Beyond Spectrogram Loss in Differentiable Time-Frequency Analysis - Archive ouverte HAL Access content directly
Journal Articles Journal of the Audio Engineering Society Year : 2023

Mesostructures: Beyond Spectrogram Loss in Differentiable Time-Frequency Analysis

Mésostructures : au-delà de la perte spectrale en analyse temps-fréquence différentiable

Abstract

Computer musicians refer to mesostructures as the intermediate levels of articulation between the microstructure of waveshapes and the macrostructure of musical forms. Examples of mesostructures include melody, arpeggios, syncopation, polyphonic grouping, and textural contrast. Despite their central role in musical expression, they have received limited attention in recent applications of deep learning to the analysis and synthesis of musical audio. Currently, autoencoders and neural audio synthesizers are only trained and evaluated at the scale of microstructure: i.e., local amplitude variations up to 100 milliseconds or so. In this paper, we formulate and address the problem of mesostructural audio modeling via a composition of a differentiable arpeggiator and time-frequency scattering. We empirically demonstrate that time-frequency scattering serves as a differentiable model of similarity between synthesis parameters that govern mesostructure. By exposing the sensitivity of short-time spectral distances to time alignment, we motivate the need for a time-invariant and multiscale differentiable time-frequency model of similarity at the level of both local spectra and spectrotemporal modulations.
Fichier principal
Vignette du fichier
JAES_2023___Mesostructures__Beyond_Spectrogram_Loss_in_Differentiable_Time_Frequency_Analysis (1).pdf (8.7 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04118474 , version 1 (06-06-2023)

Identifiers

  • HAL Id : hal-04118474 , version 1

Cite

Cyrus Vahidi, Han Han, Changhong Wang, Mathieu Lagrange, György Fazekas, et al.. Mesostructures: Beyond Spectrogram Loss in Differentiable Time-Frequency Analysis. Journal of the Audio Engineering Society, 2023, 71 (9), pp.577-585. ⟨hal-04118474⟩
45 View
47 Download

Share

Gmail Mastodon Facebook X LinkedIn More