Convergence to equilibrium of underdamped Langevin dynamics is studied under general assumptions on the potential $U$ allowing for singularities. By modifying the direct approach to convergence in $L^2$ pioneered by F. H\'erau and developped by Dolbeault, Mouhot and Schmeiser, we show that the dynamics converges exponentially fast to equilibrium in the topologies $L^2(d\mu)$ and $L^2(W^* d\mu)$, where $\mu$ denotes the invariant probability measure and $W^*$ is a suitable Lyapunov weight. In both norms, we make precise how the exponential convergence rate depends on the friction parameter $\gamma$ in Langevin dynamics, by providing a lower bound scaling as $\min(\gamma, \gamma^{-1})$. The results hold for usual polynomial-type potentials as well as potentials with singularities such as those arising from pairwise Lennard-Jones interactions between particles.