We revisit the $\eta$-pairing states in Hubbard models and explore their connections to quantum many-body scars to discover a universal scars mechanism. $\eta$-pairing occurs due to an algebraic structure known as a Spectrum Generating Algebra (SGA), giving rise to equally spaced towers of eigenstates in the spectrum. We generalize the original $\eta$-pairing construction and show that several Hubbard-like models on arbitrary graphs exhibit SGAs, including ones with disorder and spin-orbit coupling. We further define a Restricted Spectrum Generating Algebra (RSGA) and give examples of perturbations to the Hubbard-like models that preserve an equally spaced tower of the original model as eigenstates. The states of the surviving tower exhibit a sub-thermal entanglement entropy, and we analytically obtain parameter regimes for which they lie in the bulk of the spectrum, showing that they are exact quantum many-body scars. The RSGA framework also explains the equally spaced towers of eigenstates in several well-known models of quantum scars, including the AKLT model.