SARS-CoV-2 virus has spread over the world rapidly creating one of the largest pandemics ever. The absence of immunity, presymptomatic transmission, and the relatively high level of virulence of the COVID-19 infection led to a massive flow of patients in intensive care units (ICU). This unprecedented situation calls for rapid and accurate mathematical models to best inform public health policies. We develop an original parsimonious discrete-time model that accounts for the effect of the age of infection on the natural history of the disease. Analysing the ongoing COVID-19 in France as a test case, through the publicly available time series of nationwide hospital mortality and ICU activity, we estimate the value of the key epidemiological parameters and the impact of lock-down implementation delay. This work shows that including memory-effects in the modelling of COVID-19 spreading greatly improves the accuracy of the fit to the epidemiological data. We estimate that the epidemic wave in France started on Jan 20 [Jan 12, Jan 28] (95% likelihood interval) with a reproduction number initially equal to 2.99 [2.59, 3.39], which was reduced by the national lock-down started on Mar 17 to 24 [21, 27] of its value. We also estimate that the implementation of the latter a week earlier or later would have lead to a difference of about respectively −13k and +50k hospital deaths by the end of lock-down. The present parsimonious discrete-time framework constitutes a useful tool for now-and forecasting simultaneously community incidence and ICU capacity strain.