Speaker verification is a key technology in many services and applications like smartphonesand intelligent digital assistants. These applications usually require users to transmit theirrecordings, features, or models from their voices over untrusted public networks whichstored and processed them on cloud‐based infrastructure. Furthermore, the voice signalcontains a great deal of the speaker's personal and private information which raises severalprivacy issues. Therefore, it is necessary to develop speaker verification systemsthat protectthe user's voice against such threats. Herein, the cancellable biometric systems have beenintroduced as a privacy‐preserving solution. A cancellable method for speaker verificationsystemsis proposed using speakeri‐vector embeddings.This method includestwo stages: (i)i‐vector binarisation and (ii) the protection of the binary i‐vector with a shuffling schemederived from a user‐specific key. Privacy evaluation of this method according to the standards of biometric information protection (ISO/IEC 24745) shows that the proposedcancellable speaker verification system achieves the revocability, unlinkability, and irreversibility requirements. Moreover, the cancellable system improves biometric performancecompared with the unprotected system and makes it resistant to different attack scenarios.Additionally, we demonstrate that this method can also operate to protect deep neuralnetwork speaker embeddings such as x‐vectors.