Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Constructive Approximation Année : 2023

Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions

Résumé

We study the approximation of univariate functions by combining tensorization of functions with tensor trains (TTs)—a commonly used type of tensor networks (TNs). Lebesgue $L^p$-spaces in one dimension can be identified with tensor product spaces of arbitrary order through tensorization. We use this tensor product structure to define different approximation tools and corresponding approximation spaces of TTs, associated with different measures of complexity. The approximation tools are shown to have (near to) optimal approximation rates for functions with classical Besov smoothness. We then use classical interpolation theory to show that a scale of interpolated smoothness spaces is continuously embedded into the scale of TT approximation spaces and, vice versa, we show that the TT approximation spaces are, in a sense, much larger than smoothness spaces when the depth of the tensor network is not restricted but are embedded into a scale of interpolated smoothness spaces if one restricts the depth. The results of this work can be seen as both an analysis of the approximation spaces of a type of TNs and a study of the expressivity of a particular type of neural networks (NNs)—namely feed-forward sum-product networks with sparse architecture. We point out interesting parallels to recent results on the expressivity of rectifier networks.

Dates et versions

hal-04385174 , version 1 (10-01-2024)

Identifiants

Citer

Mazen Ali, Anthony Nouy. Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions. Constructive Approximation, 2023, 58 (2), pp.463-544. ⟨10.1007/s00365-023-09620-w⟩. ⟨hal-04385174⟩
19 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More