We finely describe the "coming down from infinity" for birth and death processes which eventually become extinct. Our biological motivation is to study the decrease of regulated populations which are initially large. Under general assumptions on the birth and death rates, we describe the behavior of the hitting time of large integers. We let two regimes appear and derive an expression of the speed of coming down from infinity. In the case of death rates with regular variations, we also get a central limit theorem and the asymptotic probability of extinction in small times. Finally, we apply our results to birth and death processes in varying environment in whose the environment influences the competition.