How and when did galaxies form and assemble their stars and stellar mass? The answer to these questions, so crucial to astrophysics and cosmology, requires the full reconstruction of the so-called cosmic star formation rate density (SFRD), i.e., the evolution of the average star formation rate per unit volume of the universe. While the SFRD has been reliably traced back to 10–11 billion years ago, its evolution is still poorly constrained at earlier cosmic epochs, and its estimate is mainly based on galaxies luminous in the ultraviolet and with low obscuration by dust. This limited knowledge is largely due to the lack of an unbiased census of all types of star-forming galaxies in the early universe. We present a new approach to finding dust-obscured star-forming galaxies based on their emission at radio wavelengths coupled with the lack of optical counterparts. Here, we present a sample of 197 galaxies selected with this method. These systems were missed by previous surveys at optical and near-infrared wavelengths, and 22 of them are at very high redshift (i.e., z > 4.5). The contribution of these elusive systems to the SFRD is substantial and can be as high as 40% of the previously known SFRD based on UV-luminous galaxies. The mere existence of such heavily obscured galaxies in the first two billion years after the Big Bang opens new avenues to investigate the early phases of galaxy formation and evolution, and to understand the links between these systems and the massive galaxies that ceased their star formation at later cosmic times.