Cache-based side-channel attacks (SCAs) are becoming a security threat to the emerging computing platforms. To mitigate these attacks, numerous countermeasures have been proposed. However, these countermeasures require either radical hardware modification or they are incompatible with the performance features like super-page and data de-duplication. This paper presents a countermeasure, called Flush+Prefetch, which obfuscates the memory access behavior of a secure application using independent threads that randomly access the memory belonging to secure application. Unlike existing state-of-the-art countermeasures, Flush+Prefetch works with commodity hardware and it is compatible with existing performance features. As a proof-of-concept, we have studied the effectiveness of Flush+Prefetch by defending the secret key of RSA cryptosystem against a high-resolution cache side-channel attack called Flush+Reload. We have evaluated the confidentiality of RSA decryption process on an Intel Xeon E5-2643 processor by generating 100, 000 requests to a web-server sequentially while considering the effect on performance as well. Our experimental results show that the confidentiality of memory accesses by RSA is preserved under Flush+Prefetch countermeasure. Our results show that the performance, in terms of average execution time, is improved by 10.2% for best design case as compared to the system under attack.