We establish spectral theorems for random walks on mapping class groups of connected, closed, oriented, hyperbolic surfaces, and on $\text{Out}(F_N)$. In both cases, we relate the asymptotics of the stretching factor of the diffeomorphism/automorphism obtained at time $n$ of the random walk to the Lyapunov exponent of the walk, which gives the typical growth rate of the length of a curve -- or of a conjugacy class in $F_N$ -- under a random product of diffeomorphisms/automorphisms. In the mapping class group case, we first observe that the drift of the random walk in the curve complex is also equal to the linear growth rate of the translation lengths in this complex. By using a contraction property of typical Teichm\"uller geodesics, we then lift the above fact to the realization of the random walk on the Teichm\"uller space. For the case of $\text{Out}(F_N)$, we follow the same procedure with the free factor complex in place of the curve complex, and the outer space in place of the Teichm\"uller space. A general criterion is given for making the lifting argument possible.