Post-Quantum Cryptography (PQC) will become soon the standard for many systems of the future. With the advent of quantum computers, all encrypted communications based on traditional asymmetric cryptography (e.g., RSA, ECC) will become insecure. The definition the PQC standards is an on going process proceeding at a fast pace, involving new and largely unexplored cryptographic primitives. For this reason, the design of hardware implementations of PQC algorithms is still under study. In this paper, we introduce the fundamentals of PQC, with a focus on lattice-based cryptography and its hardware security issues, namely side-channel and fault-based attacks. Then, we focus on isogeny-based cryptography and the SIKE algorithm. We highlight the importance of fault-tolerant design choices through the presentation of a fault attack, based on the electromagnetic injection of transient faults, targeting this cryptographic primitive. Finally, we show an interesting idea that starts from the observation that some PQC algorithms have an intrinsic probabilistic behavior. We argue that this characteristic is a clear opportunity that paves the way for the application of approximate (or inexact) computing to the implementation of PQC cryptography.