We study how large fluctuations are spatially correlated in the presence of quantum diffusion during inflation. This is done by computing real-space correlation functions in the stochastic-$\delta N$ formalism. We first derive an exact description of physical distances as measured by a local observer at the end of inflation, improving on previous works. Our approach is based on recursive algorithmic methods that consistently include volume-weighting effects. We then propose a "large-volume'' approximation under which calculations can be done using first-passage time analysis only, and from which a new formula for the power spectrum in stochastic inflation is derived. We then study the full two-point statistics of the curvature perturbation. Due to the presence of exponential tails, we find that the joint distribution of large fluctuations is of the form $P(\zeta_{R_1}, \zeta_{R_2}) = F(R_1,R_2,r) P(\zeta_{R_1})P( \zeta_{R_2})$, where $\zeta_{R_1}$ and $\zeta_{R_2}$ denote the curvature perturbation coarse-grained at radii $R_1$ and $R_2$, around two spatial points distant by $r$. This implies that, on the tail, the reduced correlation function, defined as $P(\zeta_{R_1}>\zeta_{\rm{c}}, \zeta_{R_2}>\zeta_{\rm{c}})/[P(\zeta_{R_1}>\zeta_{\rm{c}}) P(\zeta_{R_2}>\zeta_{\rm{c}})]-1$, is independent of the threshold value $\zeta_{\rm{c}}$. This contrasts with Gaussian statistics where the same quantity strongly decays with $\zeta_{\rm{c}}$, and shows the existence of a universal clustering profile for all structures forming in the exponential tails. Structures forming in the intermediate (i.e. not yet exponential) tails may feature different, model-dependent behaviours.