Energy efficient computing has become the key to enable the portability of new applications onto mobile devices which need to be always smaller and more powerful. As the technology node shrinks, the leakage current increases exponentially in deep submicron CMOS, so that new strategies are required in integrated systems to save power without limiting processing performances. One of the solutions is to rely on Non-Volatile Memories (NVM) and their integration within complex computing systems, but the association of heterogeneous technologies remains a real challenge. In this paper, we describe a fully embedded System-on-Chip (SoC), i.e., without external memory interface. We discuss the benefits of embedding NVM elements into the system in terms of power consumption and functionality enhancements compared to an equivalent system relying on standard volatile memory blocks. We depict the complete design from the block-diagram down to the layout of the fully functional non-volatile SoC. Our methodology includes the conception of a single bit memory cell up to the benchmarking of the architecture, in a hybrid magnetic/CMOS low-power technology, regarding the industrial constraints past experimentation in the aim to reach the quality of commercial products. We present precise pre-silicon performance estimations, using different configurations of compression algorithms as reference benchmarks that show where energy is mainly consumed.