The spermatozoon is a highly differentiated cell with unique characteristics: it is mobile, thanks to its flagellum, and is very compact. The sperm cytoplasm is extremely reduced, containing no ribosomes, and therefore does not allow translation, and its nucleus contains very closed chromatin, preventing transcription. This DNA compaction is linked to the loss of nucleosomes and the replacement of histones by protamines. Based on these characteristics, sperm was considered to simply deliver paternal DNA to the oocyte. However, some parts of the sperm DNA remain organized in a nucleosomal format, and bear epigenetic information. In addition, the nucleus and the cytoplasm contain a multitude of RNAs of different types, including non-coding RNAs (ncRNAs) which also carry epigenetic information. For a long time, these RNAs were considered residues of spermatogenesis. After briefly describing the mechanisms of compaction of sperm DNA, we focus this review on the origin and function of the different ncRNAs. We present studies demonstrating the importance of these RNAs in embryonic development and transgenerational adaptation to stress. We also look at other epigenetic marks, such as DNA methylation or post-translational modifications of histones, and show that they are sensitive to environmental stress and transmissible to offspring. The post-fertilization role of certain sperm-borne proteins is also discussed.