We investigate the quantum recoil force acting on an excited atom close to the surface of a nonreciprocal photonic topological insulator (PTI). The main atomic emission channel is the unidirectional surface plasmon propagating at the PTI-vacuum interface, and we show that it enables a spontaneous lateral recoil force that scales at short distances as 1/d^4, where d is the atom-PTI separation. Remarkably, the sign of the recoil force is polarization and orientation independent, and it occurs in a translation-invariant homogeneous system in thermal equilibrium. Surprisingly, the recoil force persists for very small values of the gyration pseudovector, which, for a biased plasma, corresponds to very low cyclotron frequencies. The ultrastrong recoil force is rooted in the quasihyperbolic dispersion of the surface plasmons. We consider both an initially excited atom and a continuous pump scenario, the latter giving rise to a steady lateral force whose direction can be changed at will by simply varying the orientation of the biasing magnetic field. Our predictions may be tested in experiments with cold Rydberg atoms and superconducting qubits.