We study the crystallization of a colloidal model system in presence of secondary nanoparticles acting as impurities. Using confocal microscopy, we show that the nanoparticles segregate in the grain boundaries of the colloidal polycrystal. We demonstrate that the texture of the polycrystal can be tuned by varying independently the nanoparticle volume fraction and the crystallization rate, and quantify our findings using standard models for the nucleation and growth of crystalline materials. Remarkably, we find that the efficiency of the segregation of the nanoparticles in the grain-boundaries is determined solely by the typical size of the crystalline grains.