We consider the stochastic heat equation on $\mathbb R^d$ with multiplicative space-time white noise noise smoothed in space. For $d\geq 3$ and small noise intensity, the solution is known to converge to a strictly positive random variable as the smoothing parameter vanishes. In this regime, we study the rate of convergence and show that the pointwise fluctuations of the smoothened solutions as well as that of the underlying martingale of the Brownian directed polymer converge to a Gaussian limit.