In this French longitudinal study, we assessed judgment of the passage of time in current life and the predictors of this judgment 2 years after the onset of the COVID-19 pandemic, i.e., at a time when there was no lockdown and no protective measures. We then compared these measures with the same participants’ passage-of-time judgments assessed during each of the past three French lockdowns. We also assessed their memory representations of the passage of time in the past, i.e., for the various lockdowns. The results showed the persistence of the feeling of time slowing down outside of lockdown. However, this was no longer linked to external factors (lack of activity, disruption of everyday routines) as found in the previous studies conducted during the lockdowns, but to an individual internal factor, namely a high level of depression in the general population. Moreover, the results revealed that the experience of the passage of time for the past lockdowns was compressed in memory, being judged to be faster than it actually was. This time compression tended to be greater in depressed people. It was also associated with a positive bias for all the other examined factors (e.g., sleep quality, life routine, boredom, happiness). We assumed that this time compression would be related to processes involved in the recall of unfolding events, with certain moments being omitted or forgotten during recall, as well as to the process of reconstruction in autobiographical memory. Our study therefore shows the long-lasting effect of lockdowns on mental health of the general population, which was expressed by the persistent feeling of a slowing down of time. It is therefore necessary to take care of this psychologically fragile population and to avoid further lockdowns in response to a new health crisis, that they cannot cope with.