This work develops further a probabilist approach to the asymptotic behavior of growth-fragmentation semigroups via the Feynman-Kac formula, which was introduced in a joint article with A.R. Watson [4]. Here, it is first shown that the sufficient condition for a Malthusian behavior which was established in [4], is also necessary. We then provide a simple criterion to ensure exponential speed of convergence, which enables us to treat cases than were not covered previously in the literature.