The main challenge of exoplanet high-contrast imaging (HCI) is to separate the signal of exoplanets from their host stars, which are many orders of magnitude brighter. HCI for ground-based observations is further exacerbated by speckle noise originating from perturbations in the Earth's atmosphere and imperfections in the telescope optics. Various data post-processing techniques are used to remove this speckle noise and reveal the faint planet signal. Often, however, a significant part of the planet signal is accidentally subtracted together with the noise. In the present work, we use explainable machine learning to investigate the reason for the loss of the planet signal for one of the most used post-processing methods: Principal Component Analysis (PCA). We find that PCA learns the shape of the telescope point spread function for high numbers of PCA components. This representation of the noise captures not only the speckle noise, but also the characteristic shape of the planet signal. Building upon these insights, we develop a new post-processing method (4S) that constrains the noise model to minimize this signal loss. We apply our model to 11 archival HCI datasets from the VLT-NACO instrument in the L'-band and find that our model consistently outperforms PCA. The improvement is largest at close separations to the star ($\leq 4 λ/D$) providing up to 1.5 magnitudes deeper contrast. This enhancement enables us to detect the exoplanet AF Lep b in data from 2011, 11 years before its subsequent discovery. We present updated orbital parameters for this object.