The initial substrate inhibiting island growth and the formation of an interfacial layer with uncontrollable characteristics are the two main drawbacks of the Atomic Layer Deposition (ALD) of high-k metal-oxide gate dielectrics on silicon (Si). In this paper, we investigate the ALD of Al2O3 films from trimethyl aluminum and H2O, on fluorhydric acid (HF) cleaned, as well as on HF-cleaned and in situ N2-NH3 plasma pretreated Si between 0 and 75 cycles. The films and their interface were characterized via Scanning Transmission Electron Microscopy coupled to Energy-Dispersive X-ray spectroscopy. The initial deposition is clearly increased on the pretreated surfaces, obtaining a linear ALD regime even after 5 ALD cycles, compared to several tens of cycles needed on HF-cleaned Si. Furthermore, a SixNy layer is formed by the N2-NH3 plasma pretreatment, which acts as a barrier layer, reducing the oxidation of the Si substrate beneath it. This analysis provides a general framework for the understanding and determination of adequate surface pretreatments, able to combat the substrate inhibited initial growth and the Si oxidation during metal-oxide ALD on Si.