Biometric systems suffer from non-revocability. In this paper, we propose a cancelable speaker verification system based on classical Gaussian Mixture Models (GMM) methodology enriched with the desired characteristics of revocability and privacy. The GMM model is transformed into a binary vector that is used by a shuffing scheme to generate a cancelable template and to guarantee the cancelability of the overall system. Leveraging the shuffing scheme, the speaker model can be rovoked and another model can be reissued. Our proposed method enables the generation of multiple cancelable speaker templates from the same biometric modality that cannot be linked to the same user. The proposed system is evaluated on the RSR2015 databases. Its outperforms the basic GMM system and experimentation show significant improvement in the speaker verification performance that achieves an Equal Error Rate (ERR) of 0.01%