A stochastic optimization technique for hyperparameter tuning in reservoir computing - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Neurocomputing Année : 2024

A stochastic optimization technique for hyperparameter tuning in reservoir computing

Résumé

This paper presents a new approach to reservoir computing (RC) optimization. Reservoir computing (RC) is a framework for building recurrent neural networks (RNNs) that alleviates the well-known learning difficulties in such neural networks by training only the output layer. While the weights of the input and the nonlinear hidden layers are randomly generated, the adjustment of critical hyperparameters, such as the input and the feedback scaling factor, is essential for optimal performance in RC. While recent hardware implementations of RC are crucial for high-speed processing, standard gradient-based hyperparameter optimization is often irrelevant due to potentially uncertain or time-varying internal functions and parameters. In this work, we propose and analyze a stochastic optimization approach using gradient approximations based solely on noisy measurements of the loss function to circumvent this problem. Our numerical and experimental results confirm that the proposed method can provide near-optimal RC hyperparameters with substantial complexity reduction compared to competing methods, validating its potential for RC optimization.
Fichier principal
Vignette du fichier
elsmanuscript.pdf (1.42 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04473502 , version 1 (22-02-2024)

Identifiants

Citer

Nickson Mwamsojo, Frederic Lehmann, Kamel Merghem, Yann Frignac, Badr-Eddine Benkelfat. A stochastic optimization technique for hyperparameter tuning in reservoir computing. Neurocomputing, 2024, 574, pp.127262. ⟨10.1016/j.neucom.2024.127262⟩. ⟨hal-04473502⟩
5 Consultations
4 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More