We have revisited the climate friction scenario during the Earth's major glacial episodes of the last 800 Myr: the Late Pliocene-Pleistocene (~0-3 Ma), the Permo-Carboniferous (~260-340 Ma) and the Neoproterozoic (~750 +/- 200 Ma). In response to periodic variations in the obliquity, the redistribution of ice/water mass and the isostatic adjustment to the surface loading affect the dynamical ellipticity of the Earth. Delayed responses in the mass redistribution may introduce a secular term in the obliquity evolution, a phenomenon called `climate friction'. We analyse the obliquity-oblateness feedback using non-linear response of ice sheets to insolation forcing and layered models with Maxwell viscoelastic rheology. Since the onset of the Northern Hemisphere glaciation (~3 Ma), we predict an average drift of only ~0.01 deg Myr^-1 modulated by the main ~1.2 Myr modulating obliquity period. This value is well reproduced when high-resolution oxygen-isotope records are used to constrain the ice load history. For earlier glaciations, we find that the climate friction effect is not proportional to the amplitude of the ice-age load, as it was previously assumed. A possible increase in the non-linear response of ice sheets to insolation forcing and latitudinal changes in this forcing may strongly limit the contribution of the obliquity variations to glacial variability, and thereby the climate friction amplitude. The low-latitude glaciations of the Sturtian glacial interval (ca 700-750 Ma) have probably no influence on the obliquity, while we predict a maximal possible absolute change of ~2° for the Varanger interval (ca 570-620 Ma). We show that this mechanism cannot thus explain a substantial and rapid decrease in obliquity (of~30°) as previously suggested by D.M. Williams et al. (1998) to support the high obliquity scenario of G.E. Williams (1993). Overall, we find that climate friction cannot have changed the Earth's obliquity by more than 3-4° over the last 800 Myr.