Preemption (or the splitting of activities) is a common practice in many project environments, and has been a standard feature of commercial project management software packages for years. Despite its prevalence in daily practice, preemption has received little attention in the project scheduling literature. A possible explanation for this lack of research interest is the common assumption that preemption only has a limited impact on the optimal makespan of a project. In this article, however, we show that the benefit of preemption can be significant, and that it increases with the size and the complexity of the project network. In addition, we also investigate how activity duration variability impacts the benefits of preemption. To this end, we study the preemptive stochastic resource-constrained project scheduling problem (PSRCPSP), and present an exact solution procedure. Even though the deterministic preemptive resource-constrained project scheduling problem (PRCPSP) has received some attention in the literature, we are the first to study the PSRCPSP. We use hypoexponential distributions to model the activity durations, and define a new continuous-time Markov chain (CTMC) that drastically reduces memory requirements when compared to the well-known CTMC of Kulkarni and Adlakha (1986) (Operations Research, 34(5), 769–781). In addition, we also propose a new and efficient approach to structure the state space of the CTMC.