Our study addresses the influence of a TiO$_2$ buffer layer on the morphological, structural, and magnetic properties of Co:TiO$_2$ films grown on (001) SrTiO$_3$ substrates by RF sputtering. We demonstrate that a direct correlation exist between the morphology, the Co heterogeneity, and the magnetic properties measured in the films. Correlated analysis by cross section transmission electron microscopy, energy dispersive x-ray, and x-ray photoemission spectroscopy reveals that the Co is not uniformly distributed in the film but concentrated in the surface clusters. Atomic force microscopy analysis illustrates that the unbuffered films present a large density of surface clusters. These clusters are not metallic Co but Co rich TiO$_2$ anatase phase and they are accompanied by structural defects in the film: dislocations, small angle grain boundaries. Magnetometry analysis shows that the unbuffered films have a net ferromagnetic behavior, while in the buffered ones the ferromagnetism is quenched. Therefore, we conclude that the magnetism in unbuffered samples is related to the surface clusters and seems to have an extrinsic nature.