The uncertainty quantification for the limit of a Stochastic Approximation (SA) algorithm is analyzed. In our setup, this limit $f^*$ is defined as a zero of an intractable function and is modeled as uncertain through a parameter $\theta$. We aim at deriving the function $f^*$, as well as the probabilistic distribution of $f^*(\theta)$ given a probability distribution $\pi$ for $\theta$. We introduce the so-called Uncertainty Quantification for SA (UQSA) algorithm, an SA algorithm in increasing dimension for computing the basis coefficients of a chaos expansion of $\theta \mapsto f^*(\theta)$ on an orthogonal basis of a suitable Hilbert space. UQSA, run with a finite number of iterations $K$, returns a finite set of coefficients, providing an approximation $\widehat{f^*_K}(\cdot)$ of $f^*$. We establish the almost-sure and $L^p$-convergences in the Hilbert space of the sequence of functions $\widehat{f^*_K}(\cdot)$ when the number of iterations $K$ tends to infinity. This is done under mild, tractable conditions, uncovered by the existing literature for convergence analysis of infinite dimensional SA algorithms. For a suitable choice of the Hilbert basis, the algorithm also provides an approximation of the expectation, of the variance-covariance matrix and of higher order moments of the quantity $\widehat{f^*_K}(\theta)$ when $\theta$ is random with distribution $\pi$. UQSA is illustrated and the role of its design parameters is discussed numerically.