The polarization transfer coefficients of a relativistic magnetized plasma are derived. These results apply to any momentum distribution function of the particles, isotropic or anisotropic. Particles interact with the radiation either in a non-resonant mode when the frequency of the radiation exceeds their characteristic synchrotron emission frequency or quasi-resonantly otherwise. These two classes of particles contribute differently to the polarization transfer coefficients. For a given frequency, this dichotomy corresponds to a regime change in the dependence of the transfer coefficients on the parameters of the particle's population, since these parameters control the relative weight of the contribution of each class of particles. Our results apply to either regimes as well as the intermediate one. The derivation of the transfer coefficients involves an exact expression of the conductivity tensor of the relativistic magnetized plasma that has not been used hitherto in this context. Suitable expansions valid at frequencies much larger than the cyclotron frequency allow us to analytically perform the summation over all resonances at high harmonics of the relativistic gyrofrequency.