We study the impact of the muon pair production and double pair production processes induced by ultra-high energy photons on the cosmic microwave background. Although the muon pair production cross section is smaller than the electron pair production one, the associated energy loss length is comparable or shorter than the latter (followed by inverse Compton in the deep Klein-Nishina regime) at high-redshift, where the effect of the astrophysical radio background is expected to be negligible. By performing a simulation taking into account the details of $e/\gamma$ interactions at high energies, we show that a significant fraction of the electromagnetic energy injected at $E\gtrsim 10^{19}\,$eV at redshift $z\gtrsim 5$ is channeled into neutrinos. The double pair production plays a crucial role in enhancing the multiplicity of muon production in these electromagnetic cascades. The ultra-high energy neutrino spectrum, yet to be detected, can in principle harbour information on ultra-high energy sources in the young universe, either conventional or exotic ones, with weaker constraints from the diffuse gamma ray flux compared to their low redshift counterparts.