In approximate Kohn-Sham density-functional theory, self-interaction manifests itself as the dependence of the energy of an orbital on its fractional occupation. This unphysical behavior translates into qualitative and quantitative errors that pervade many fundamental aspects of density-functional predictions. Here, we first examine self-interaction in terms of the discrepancy between total and partial electron removal energies, and then highlight the importance of imposing the generalized Koopmans' condition -- that identifies orbital energies as opposite total electron removal energies -- to resolve this discrepancy. In the process, we derive a correction to approximate functionals that, in the frozen-orbital approximation, eliminates eliminates the unphysical occupation dependence of orbital energies up to the third order in the single-particle densities. This non-Koopmans correction brings physical meaning to single-particle energies; when applied to common local or semilocal density functionals it provides results that are in excellent agreement with experimental data -- with an accuracy comparable to that of GW many-body perturbation theory -- while providing an explicit total energy functional that preserves or improves on the description of established structural properties.