The emergence of non-volatile random access memory technologies, such as resistive and spintronic RAMs are triggering intense interdisciplinary activity. These technologies have the potential of providing many benefits, such as energy efficiency, high integration density, CMOS-compatibility, re-configurability, non-volatility and open the path towards novel computational structures and approaches, for the traditional Von-Neumann architectures and beyond. These promising characteristics, coupled with the ever-increasing limitations faced by traditional CMOS-based storage and computational structures, have driven the research community towards completely revisiting the existing computing and storage paradigms, now focusing on providing hardware solutions for in-memory and neuromorphic computing. This has resulted in an intensified research activity in the device physics, striving to achieve circuit-worth devices, reliable compact models and novel architectures. The purpose of this paper is to provide a comprehensive overview of the device physics, issues related to its use in electronic circuits, methodologies for their compact modelling and simulations, and their integration in storage and computational structures.