Timing-based side-channels play an important role in exposing the state of a process execution on underlying hardware by revealing information about timing and access patterns. Side-channel attacks (SCAs) are powerful cryptanalysis techniques that focus on the underlying implementation of cryptographic ciphers during execution rather than attacking the structure of cryptographic functions. This paper reviews cache-based software side-channel attacks, mitigation and detection techniques that target various cryptosystems, particularly RSA, proposed over the last decade (2007–2018). It provides a detailed taxonomy of attacks on RSA cryptosystems and discusses their strengths and weaknesses while attacking different algorithmic implementations of RSA. A threat model is presented based on the cache features that are being leveraged for such attacks across cache hierarchy in computing architectures. The paper also provides a classification of these attacks based on the source of information leakage. It then undertakes a qualitative analysis of secret key retrieval efficiency, complexity, and the features being exploited on target cryptosystems in these attacks. The paper also discusses the mitigation and detection techniques proposed against such attacks and classifies them based on their effectiveness at various levels in caching hardware and leveraged features. Finally, the paper discusses recent trends in attacks, the challenges involved in their mitigation, and future research directions needed to deal with side-channel information leakage.