Streaming services come with their own challenges and technical issues that still need to be addressed for satisfying the target quality of experience (QoE) of the end-users in mobile environments. In this paper, we explore the idea of combining users' context information with the packed prefetching process features to enhance users' QoE in heterogeneous networks. More specifically, we propose a scheduling mechanism for video streaming traffic, in which the access to the network resources is restricted to users with a signal-to-noise plus interference ratio (SINR) above a given threshold. This scheme benefits from the fact that, as users are in permanent motion, they may experience different SINR values during the same video streaming session offering the opportunity to only schedule users with good channel conditions. The proposed scheduling approach (subsequently referred to as context-aware mode switching (CAMS)) not only allows to achieve overall network spectral efficiency improvement, but also guarantees fairness and QoE among users. Our simulation results show that CAMS achieves almost 1 bit per second per Hertz gain compared to the conventional scheduler (without CAMS), and up to 87% improvement in the probability of no starvations when users move at 40 kmph.