X-ray binaries (XRBs) exhibit spectral hysteresis for luminosities in the range $10^{-2}\lesssim L/L_\mathrm{Edd}\lesssim 0.3$, with a hard X-ray spectral state that persists from quiescent luminosities up to $\gtrsim 0.3L_\mathrm{Edd}$, transitioning to a soft spectral state that survives with decreasing luminosities down to $\sim 10^{-2}L_\mathrm{Edd}$. We present a possible approach to explain this behavior based on the thermal properties of a magnetically arrested disk simulation. By post-processing the simulation to include radiative effects, we solve for all the thermal equilibrium solutions as the accretion rate, $\dot{M}$, varies along the XRB outburst. For an assumed scaling of the disk scale height and accretion speed with temperature, we find that there exists two solutions in the range of $ 10^{-3}\lesssim\dot{M}/\dot{M}_{\rm Eddington} \lesssim 0.1$ at $r=8\:r_g$ ($ 4\times10^{-2}\lesssim\dot{M}/\dot{M}_{\rm Eddington} \lesssim 0.5$ at $r=3\:r_g$) : a cold, optically thick one and a hot, optically thin one. This opens the possibility of a natural thermal hysteresis in the right range of luminosities for XRBs. We stress that our scenario for the hysteresis does not require to invoke the strong-ADAF principle nor does it require for the magnetization of the disk to change along the XRB outburst. In fact, our scenario requires a highly magnetized disk in the cold, soft state to reproduce the soft-to-hard state transition at the right luminosities. Hence, a prediction of our scenario is that there should be a jet, although possibly very weakly dissipative, in the soft state of XRBs. We also predict that if active galactic nuclei (AGN) have similar hysteresis cycles and are strongly magnetized, they should undergo a soft-to-hard state transition at much lower $L/L_\mathrm{Edd}$ than XRBs.