Controlling thermal transport in insulators and semiconductors is crucial for many technological elds such as thermoelectrics and thermal insulation, for which a low thermal conductivity (κ) is desirable. A major obstacle for realizing low κ materials is Rayleigh's law, which implies that acoustic phonons, which carry most of the heat, are insensitive to scattering by point defects at low energy. We demonstrate, with large scale simulations on tens of millions of atoms, that isotropic long-range spatial correlations in the defect distribution can dramatically reduce phonon lifetimes of important low-frequency heat-carrying modes, leading to a large reduction of κ potentially an order of magnitude at room temperature. We propose a general and quantitative framework for controlling thermal transport in complex functional materials through structural spatial correlations, and we establish the optimal functional form of spatial correlations that minimize κ. We end by briey discussing experimental realizations of various correlated structures.