Quantum dots (QDs) embedded in inorganic matrices have been extensively studied for their potential applications in lighting, displays, and solar cells. While a significant amount of research studies focused on their experimental fabrication, the origin of their relatively low photoluminescence quantum yield has not been investigated yet, although it severely hinders practical applications. In this study, we use time-dependent density functional theory (TDDFT) to pinpoint the nature of excited states of CdSe QDs embedded in various inorganic matrices. The formation of undercoordinated Se atoms and nonbridging oxygen atoms at the QD/glass interface is responsible for the localization of a hole wave function, leading to the formation of low-energy excited states with weak oscillator strength. These states provide pathways for nonradiative processes and compete with radiative emission. The photoluminescence performance is predicted for CdSe QDs in different matrices and validated by experiments. The results of this study have significant implications for understanding the underlying photophysics of CdSe QDs embedded in inorganic matrices that would facilitate the fabrication of highly luminescent glasses.