Composites materials comprising nanoparticles dispersed in a matrix are of great scientific and technological interest, since nanoparticles can enhance dramatically the matrix properties or even impart new functionalities, and because the matrix can act as a template that structures the particles at the nanoscopic level. However, controlling the three-dimensional spatial distribution of nanoparticles in a molecular or macromolecular matrix is a challenging task, as particle segregation usually depends crucially on the surface chemistry of the particles. Here, we present a model hybrid material, obtained by dispersing nanoparticles in a colloidal crystalline matrix, composed of thermoresponsive micelles. Using confocal microscopy, we show that the nanoparticles segregate in a network of thin sheets, in analogy to impurities confined in the grain boundaries of atomic polycrystals. We demonstrate that the size of the colloidal crystallites is tuned by varying independently the nanoparticle concentration (regardless of their composition and surface chemistry) and the crystallization rate, because they both determine the number of critical nuclei during the nucleation process and we quantify our findings using classical nucleation theory. Remarkably, we find that the efficiency of the segregation of the nanoparticles in the grain-boundaries is dictated solely by the typical size of the crystalline grains, due to the fact that the larger a grain can grow, the higher the concentration of the impurities progressively expelled from the crystallites during their growth and eventually trapped in the grain boundary, as we clearly show. Our method provides a general approach for confining nanoparticles in absence of any external field and in a controlled and tunable fashion in a three-dimensional soft colloidal matrix.