Nanocomposites made of crystalline nanoinclusions embedded in an amorphous matrix are at the forefront of current research for energy harvesting applications. However, the microscopic mechanisms leading alternatively to an effectively reduced or enhanced thermal transport still escape understanding. In this work, we present a molecular dynamics simulation study of model systems, where for the first time we combine a microscopic investigation of phonon dynamics with the macroscopic thermal conductivity calculation, to shed light on thermal transport in these materials. We clearly show that crystalline nanoinclusions represent a novel scattering source for vibrational waves, modifying the nature of low energy vibrations and significantly anticipating the propagative-to-diffusive crossover (Ioffe–Regel), usually located at energies of few THz in amorphous materials. Moreover, this crossover position can be tuned by changing the elastic contrast between nanoinclusions and the matrix, and anticipated by a factor as large as 10 for a harder inclusion. While the propagative contribution to thermal transport is drastically reduced, the calculated thermal conductivity is not significantly affected in the chosen system, as the diffusive contribution dominates heat transport when all phonons are thermally populated. These findings allow finally to understand the panoply of contradictory results reported on thermal transport in nanocomposites and give clear indications to the characteristics that the parent phases should have for efficiently reducing heat transport in a nanocomposite.