Communication Dans Un Congrès Année : 2024

Deep Sturm–Liouville: From Sample-Based to 1D Regularization with Learnable Orthogonal Basis Functions

Résumé

Although Artificial Neural Networks (ANNs) have achieved remarkable success across various tasks, they still suffer from limited generalization. We hypothesize that this limitation arises from the traditional sample-based (0--dimensionnal) regularization used in ANNs. To overcome this, we introduce \textit{Deep Sturm--Liouville} (DSL), a novel function approximator that enables continuous 1D regularization along field lines in the input space by integrating the Sturm--Liouville Theorem (SLT) into the deep learning framework. DSL defines field lines traversing the input space, along which a Sturm--Liouville problem is solved to generate orthogonal basis functions, enforcing implicit regularization thanks to the desirable properties of SLT. These basis functions are linearly combined to construct the DSL approximator. Both the vector field and basis functions are parameterized by neural networks and learned jointly. We demonstrate that the DSL formulation naturally arises when solving a Rank-1 Parabolic Eigenvalue Problem. DSL is trained efficiently using stochastic gradient descent via implicit differentiation. DSL achieves competitive performance and demonstrate improved sample efficiency on diverse multivariate datasets including high-dimensional image datasets such as MNIST and CIFAR-10.

Fichier principal
Vignette du fichier
main.pdf (384.57 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence
DOI

Cite 10.24432/C5XW20 Jeu de données Barry Becker, R. K. (1996). Adult [Data set]. UCI Machine Learning Repository. https://doi.org/10.24432/C5XW20

Revenu individuel (census income)
DOI

Cite 10.24432/C50S4B Jeu de données Unknown. (2020). Dry Bean [Data set]. UCI Machine Learning Repository. https://doi.org/10.24432/C50S4B

Classification de variétés de haricots

Dates et versions

hal-04446268 , version 1 (08-02-2024)
hal-04446268 , version 2 (07-04-2025)

Licence

Identifiants

  • HAL Id : hal-04446268 , version 2

Citer

David Vigouroux, Joseba Dalmau, Louis Béthune, Victor Boutin. Deep Sturm–Liouville: From Sample-Based to 1D Regularization with Learnable Orthogonal Basis Functions. International Conference on Machine Learning, Jul 2025, Vancouver (Canada), Canada. ⟨hal-04446268v2⟩
10013 Consultations
2258 Téléchargements

Partager

  • More