We investigate remote surface roughness (RSR) scattering by the SiO2/HfO2 interface in Fully Depleted Silicon-on-Insulator devices using Non-Equilibrium Green's Functions. We show that the RSR mobility is controlled by cross-correlations between the surface roughness profiles at the Si/SiO2 and SiO2/HfO2 interfaces. Therefore, surface roughness and remote surface roughness cannot be modeled as two independent mechanisms. RSR tends to enhance the total mobility when the Si/SiO2 interface and SiO2 thickness profiles are correlated, and to decrease the total mobility when they are anti-correlated. We discuss the implications for the high-j/Metal gate technologies. (C) 2015 AIP Publishing LLC.