Both the three-dimensional density of red clump giants and the gas kinematics in the inner Galaxy indicate that the pattern speed of the Galactic bar could be much lower than previously estimated. Here, we show that such slow bar models are unable to reproduce the bimodality observed in local stellar velocity space. We do so by computing the response of stars in the solar neighbourhood to the gravitational potential of slow and fast bars, in terms of their perturbed distribution function in action-angle space up to second order, as well as by identifying resonantly trapped orbits. We also check that the bimodality is unlikely to be produced through perturbations from spiral arms, and conclude that, contrary to gas kinematics, local stellar kinematics still favour a fast bar in the Milky Way, with a pattern speed of the order of almost twice (and no less than 1.8 times) the circular frequency at the Sun's position. This leaves open the question of the nature of the long flat extension of the bar in the Milky Way.