As a high-intensity laser pulse impinges a thin solid foil, high-order harmonics of the incident frequency can be observed at the rear (non-illuminated) side of this foil. Using numerical simulations, we show that these harmonics can be generated either at the front or at the rear side of the target. We analyze the mechanisms responsible for these two types of emission, and discuss their connection with those involved in the generation of harmonics detected on the front side. The combined measurements of spectra on both sides of the target constitute a powerful, but often nontrivial, probe of the ultrafast plasma dynamics.