Building galaxy merger trees from a state-of-the-art cosmological hydrodynamical simulation, Horizon-AGN, we perform a statistical study of how mergers and diffuse stellar mass acquisition processes drive galaxy morphologic properties above z > 1. By diffuse mass acquisition here, we mean both accretion of stars by unresolved mergers (relative stellar mass growth smaller than 4.5 per cent) as well as in situ star formation when no resolved mergers are detected along the main progenitor branch of a galaxy. We investigate how stellar densities, galaxy sizes and galaxy morphologies (defined via shape parameters derived from the inertia tensor of the stellar density) depend on mergers of different mass ratios. We investigate how stellar densities, effective radii and shape parameters derived from the inertia tensor depend on mergers of different mass ratios. We find strong evidence that diffuse stellar accretion and in situ formation tend to flatten small galaxies over cosmic time, leading to the formation of discs. On the other hand, mergers, and not only the major ones, exhibit a propensity to puff up and destroy stellar discs, confirming the origin of elliptical galaxies. We confirm that mergers grow galaxy sizes more efficiently than diffuse processes ($$r_{0.5}\propto M_{\rm s}^{0.85}$$ and $$r_{0.5}\propto M_{\rm s}^{0.1}$$ on average, respectively) and we also find that elliptical galaxies are more susceptible to grow in size through mergers than disc galaxies with a size–mass evolution $$r_{0.5}\propto M_{\rm s}^{1.2}$$ instead of $$r_{0.5}\propto M_{\rm s}^{-0.5}-M^{0.5}$$ for discs depending on the merger mass ratio. The gas content drives the size–mass evolution due to merger with a faster size growth for gas-poor galaxies $$r_{0.5}\propto M_{\rm s}^{2}$$ than for gas-rich galaxies r_0.5 ∝ M_s.