We investigate how the extragalactic proton component derived within the ’escape model’ can be explained by astrophysical sources. We find that the complete extragalactic proton spectrum can be explained by a single source population with late time evolution, such as BL Lacs or FR I galaxies. We then calculate the diffuse neutrino and $\gamma$-ray fluxes produced by these CR protons interacting with gas inside their sources, their host galaxies and galaxy clusters. For a spectral slope of CRs close to $\alpha =$ 2.1 - 2.2 as suggested by shock acceleration, we find that these UHECR sources contribute to a dominant fraction of both the isotropic $\gamma$-ray background and to the extragalactic part of the astrophysical neutrino signal observed by IceCub.