The propagation and maintenance of gene expression programs are at the foundation of the preservation of cell identity. A large and complex set of epigenetic mechanisms enables the long-term stability and inheritance of transcription states. A key property of authentic epigenetic regulation is being independent from the instructive signals used for its establishment. This makes epigenetic regulation, particularly epigenetic silencing, extremely robust and powerful to lock regulatory states and stabilise cell identity. In line with this, the establishment of epigenetic silencing during development restricts cell potency and maintains the cell fate choices made by transcription factors (TFs). However, how more immature cells that have not yet established their definitive fate maintain their transitory identity without compromising their responsiveness to signalling cues remains unclear. A paradigmatic example is provided by pluripotent embryonic stem (ES) cells derived from a transient population of cells of the blastocyst. Here, we argue that ES cells represent an interesting "epigenetic paradox": even though they are captured in a self-renewing state characterised by extremely efficient maintenance of their identity, which is a typical manifestation of robust epigenetic regulation, they seem not to heavily rely on classical epigenetic mechanisms. Indeed, self-renewal strictly depends on the TFs that previously instructed their undifferentiated identity and relies on a particular signalling-dependent chromatin state where repressive chromatin marks play minor roles. Although this "epigenetic paradox" may underlie their exquisite responsiveness to developmental cues, it suggests that alternative mechanisms to faithfully propagate gene regulatory states might be prevalent in ES cells.