The effect of deposition conditions and further annealing treatment on microstructure and optical properties of (Si,Er)-codoped HfO2 thin films is investigated. The films are grown on silicon substrates by RF magnetron co-sputtering of Si and erbium oxide pellets on a HfO2 target. As-deposited and annealed samples are examined by means of spectroscopic ellipsometry, Fourier transform infrared spectroscopy and photoluminescence method. It is demonstrated that the variation of RF power density allows monitoring the dopant content. An annealing of the samples at 800–1100 °C for 10–60 min in nitrogen atmosphere results in the phase separation and the formation of HfO2, SiO2 and pure silicon phases. The films annealed at 900–950 °C demonstrate the red luminescence that is ascribed to the carrier recombination in Si nanocrystals. Annealing at higher temperatures causes an enhancement of rare-earth luminescence under non-resonant excitation providing an additional argument toward the formation of silicon nanoclusters. The mechanism of the excitation of Er ions is found to be similar to that of Si-rich-SiO2 films doped with rare-earths. The Si nanocrystals are considered as the main sensitizer at visible excitation while the energy transfer from host defects dominates at ultraviolet-deep blue illumination.