We finely describe the speed of "coming down from infinity" for birth and death processes which eventually become extinct. Under general assumptions on the birth and death rates, we firstly determine the behavior of the successive hitting times of large integers. We put in light two different regimes depending on whether the mean time for the process to go from $n+1$ to $n$ is negligible or not compared to the mean time to reach $n$ from infinity. In the first regime, the coming down from infinity is very fast and the convergence is weak. In the second regime, the coming down from infinity is gradual and a law of large numbers and a central limit theorem for the hitting times sequence hold. By an inversion procedure, we deduce that the process is a.s. equivalent to a non-increasing function when the time goes to zero. Our results are illustrated by several examples including applications to population dynamics and population genetics. The particular case where the death rate varies regularly is studied in details.