Periodic Signal Recovery with Regularized Sine Neural Networks - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Periodic Signal Recovery with Regularized Sine Neural Networks

David Robin
  • Fonction : Auteur
  • PersonId : 1216866
  • IdHAL : robindar
Kevin Scaman
  • Fonction : Auteur
  • PersonId : 1062981
Marc Lelarge
  • Fonction : Auteur
  • PersonId : 833445

Résumé

We consider the problem of learning a periodic one-dimensional signal with neural networks, and designing models that are able to extrapolate the signal well beyond the training window. First, we show that multi-layer perceptrons with ReLU activations are provably unable to perform this extrapolation task, and lead to poor performance in practice even close to the training window. Then, we propose a modified training procedure for two-layer architectures with sine activations with a more diverse feature initialization and well-chosen non-convex regularization, that is able to extrapolate the signal with low error well beyond the training window. This procedure yields results several orders of magnitude better than its competitors for distant extrapolation (beyond 100 periods of the signal), while being able to accurately recover the frequency spectrum of the signal in a multi-tone setting.
Fichier principal
Vignette du fichier
60_periodic_signal_recovery_with_.pdf (3.89 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03896115 , version 1 (06-01-2023)

Identifiants

  • HAL Id : hal-03896115 , version 1

Citer

David Robin, Kevin Scaman, Marc Lelarge. Periodic Signal Recovery with Regularized Sine Neural Networks. NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations, Dec 2022, New Orleans, United States. ⟨hal-03896115⟩
31 Consultations
69 Téléchargements

Partager

Gmail Facebook X LinkedIn More