We carry out the construction of some ill-posed multiplicative stochastic heat equations on unbounded domains. The two main equations our result covers are, on the one hand the parabolic Anderson model on $\mathbf{R}^3$, and on the other hand the KPZ equation on $\mathbf{R}$ via the Cole-Hopf transform. To perform these constructions, we adapt the theory of regularity structures to the setting of weighted Besov spaces. One particular feature of our construction is that it allows one to start both equations from a Dirac mass at the initial time.