In recent years, the manipulation of Fano resonances in the time domain has unlocked deep insights into a broad spectrum of systems’ coherent dynamics. Here, inelastic scattering of light with coherent acoustic phonons is harnessed to achieve complex Fano resonances. The sudden change of phonon momentum during reflection leads to a transition from anti-Stokes to Stokes light scattering, producing two different resonances that interfere in the measurement process. We highlight the conditions necessary to achieve such interference, revealing an underlying symmetry between photons and phonons, and verify the theory experimentally. Then, we demonstrate the possibility to characterize energy and coherence losses at rough interfaces, thus providing a mechanism for nondestructive testing of interface quality. Our results describe numerous unexplained observations in ultrafast acoustics and can be generalized to the scattering of light with any waves.