Deep neural networks (DNNs) are shown to be vulnerable to adversarial attacks-- carefully crafted additive noise that undermines DNNs integrity. Previously proposed defenses against these attacks require substantial overheads, making it challenging to deploy these solutions in power and computational resource-constrained devices, such as embedded systems and the Edge. In this paper, we explore the use of voltage overscaling (VOS) as a lightweight defense against adversarial attacks. Specifically, we exploit the stochastic timing violations of VOS to implement a moving-target defense for DNNs. Our experimental results demonstrate that VOS guarantees effective defense against different attack methods, does not require any software/hardware modifications, and offers a by-product reduction in power consumption.