Species ranges are dynamic, experiencing expansions, contractions or shifts as a response to habitat changes induced by extrinsic factors such as climate change and, more recently, human activities. While the scientific literature has explored the genetic effects of spatial processes, published studies rarely incorporate life-history traits to study the effect of such changes on species living in the same environments. There is thus a gap in our understanding regarding the variation in genetic diversity patterns among species with distinct life-history traits such as growth rates and generation times, experiencing the same habitat change scenarios. In this study, we first used spatial simulations to investigate the temporal dynamics of genetic diversity within refugium populations experiencing a range expansion followed by a stationary and a contraction period. We explored different scenarios, varying both the speed of contraction and the life-history traits of the simulated species. In addition, we used a simpler panmictic model for which we derived analytical results. Altogether, we identified three temporal dynamics of genetic diversity in the refugium population during the contraction phase: scenarios where genetic diversity i) decreased throughout the contractions, ii) increased for periods that could be greater than thousands of years before plateauing and then decreasing or iii) followed a persistent increasing trend, without any visible effect of the expansion or contraction. We show that these different temporal dynamics can be predicted by comparing the observed expected heterozygosity (He) to the values expected if the species were at equilibrium within the refuge (He refuge) and within the whole landscape (He landscape). We also observe that there are scenarios where a rapid contraction maintains more diversity just at the end of the contraction, as widely believed and as reported in a previous simulation study. However, we also observe the opposite pattern for a wide range of parameters. The widespread idea that observing high diversity levels in a refugium population is due to a recent and rapid habitat loss is thus not necessarily true and will depend on various life history traits and how they relate to habitat change dynamics.