The general decomposition of the spectral correlation tensor Rij(k) by Cambon et al. [J. Fluid Mech. 202, 295 (1989) ; 337, 303 (1997) ] into directional and polarization components is applied to the representation of Rij(k) by spherically averaged quantities. The decomposition splits the deviatoric part Hij(k) of the spherical average of Rij(k) into directional and polarization components Hij(e)(k) and Hij(z)(k). A self-consistent representation of the spectral tensor is constructed in terms of these spherically averaged quantities. The directional and polarization components must be treated independently: representation of the spectral tensor using the spherical average Hij(k) alone proves to be inconsistent with Navier-Stokes dynamics. In particular, a spectral tensor consistent with a prescribed Reynolds stress is not unique. Since spherical averaging entails a loss of information, the description of an anisotropic correlation tensor by spherical averages is limited to weak departures from isotropy. The degree of anisotropy permitted is restricted by realizability requirements. More general descriptions can be given using a higher-order expansion of the spectral tensor. Directionality is described by a conventional expansion in spherical harmonics, but polarization requires an expansion in special tensorial quantities generated by irreducible representations of the rotation group SO3. These expansions are considered in more detail in the special case of axial symmetry.