The present work aims at developing a spectral model for a passive scalar field and its associated scalar flux in homogeneous anisotropic turbulence. This is achieved using the paradigm of eddy-damped quasi-normal markovian (EDQNM) closure extended to anisotropic flows. In order to assess the validity of this approach, the model is compared to several detailed DNS and experiments of shear-driven flows and isotropic turbulence with a mean scalar gradient at moderate Reynolds numbers. This anisotropic modelling is then used to investigate the passive scalar dynamics at very high Reynolds numbers. In the framework of homogeneous isotropic turbulence submitted to a mean scalar gradient , decay and growth exponents for the cospectrum and scalar energies are obtained analytically and assessed numerically thanks to EDQNM closure. With the additional presence of a mean shear, the scaling of the scalar flux and passive scalar spectra in the inertial range are investigated and confirm recent theoretical predictions. Finally, it is found that in shear-driven flows, the small scales of the scalar second-order moments progressively return to isotropy when the Reynolds number increases.