The magma ocean was a important reservoir for Earth's primary volatiles. Understanding the volatile fluxes between the early atmosphere and the magma ocean is fundamental for quantifying the volatile budget of our planet. Here we investigate the vaporization of carbon and hydrogen at the boundary between the magma ocean and the thick, hot early atmosphere using first-principles molecular dynamics calculations. We find that carbon is rapidly devolatilized, while hydrogen mostly remains dissolved in the magma during the existence of a thick silicate-bearing atmosphere. In the early stages of the magma ocean, the atmosphere would have contained significantly more carbon than hydrogen, and the high concentrations of carbon dioxide would have prolonged the cooling time of early Earth.