Fault zones are associated with multi-scale heterogeneities of rock properties. Large scale variations may be imaged with conventional seismic reflection methods that detect offsets in geological units, and tomographic techniques that provide average seismic velocities in resolved volumes. However, characterizing elementary localized inhomogeneities of fault zones, such as cracks and fractures, constitutes a challenge for conventional techniques. Resolving these small-scale heterogeneities can provide detailed information for structural and mechanical models of fault zones. Recently, the reflection matrix approach utilizing body wave reflections in ambient noise cross-correlations was extended with the introduction of aberration corrections to handle the actual lateral velocity variations in the fault zone~\cite{touma2021}. Here this method is applied further to analyze the distribution of scatterers in the first few kilometers of the crust in the San Jacinto Fault Zone at the Sage Brush Flat (SGB) site, southeast of Anza, California. The matrix approach allows us to image not only specular reflectors but also to resolve the presence, location and intensity of scatterers of seismic waves starting with a simple homogeneous background velocity model of the medium. The derived three-dimensional image of the fault zone resolves lateral variations of scattering properties in the region within and around the surface fault traces, as well as differences between the Northwest (NW) and the Southeast (SE) parts of the study area. A localized intense damage zone at depth is observed in the SE section, suggesting that a geometrical complexity of the fault zone at depth induces ongoing generation of rock damage.