A new framework to track the structure of temporal networks with a signal processing approach is introduced. The method is based on the duality between static networks and signals, obtained using a multidimensional scaling technique, that makes possible the study of the network structure from frequency patterns of the corresponding signals. In this paper, we propose an approach to identify structures in temporal networks by extracting the most significant frequency patterns and their activation coefficients over time, using nonnegative matrix factorization of the temporal spectra. The framework, inspired by audio decomposition, allows transforming back these frequency patterns into networks, to highlight the evolution of the underlying structure of the network over time. The effectiveness of the method is first evidenced on a synthetic example, prior being used to study a temporal network of face-to-face contacts. The extracted sub-networks highlight significant structures decomposed on time intervals that validates the relevance of the approach on real-world data.