Recent advances have prominently highlighted physics informed neural networks (PINNs) as an efficient methodology for solving partial differential equations (PDEs). The present paper proposes a proof of concept exploring the use of PINNs as an alternative to finite element (FE) solvers in both classical and gradient-enhanced solid mechanics. To this end, spatio-temporal PINNs are designed to represent continuous solutions of boundary value problems within spatio-temporal space. These PINNs directly incorporate the equilibrium and constitutive equations in their differential and rate forms, bypassing the requirement for incremental implementation. This simplifies application of PINNs to solve complex mechanical problems, particularly those involved in the context of gradient-enhanced continua. Moreover, traditional meshing is no longer required as it is replaced by a point cloud, making it possible to overcome meshing drawbacks. The results of this investigation prove the effectiveness of the proposed methodology, especially with regards to non-monotonic loading conditions and irreversible plastic deformation. Compared to classical FE approaches, the proposed spatio-temporal PINNs are more readily applied to complex problems, which are tackled in their raw form. This is especially true for gradient-enhanced continuum problems, where there is no need to introduce additional degrees of freedom as in classical FE approaches. However, PINNs training generally requires more computation time, a challenge that can be mitigated by employing the concept of transfer learning as shown in this paper. This concept, which is very useful when performing parametric studies, involves applying knowledge grained from solving one problem to another different but related one. The use of PINNs as mechanical solvers is shown to be highly promising in the forthcoming era, where advancements in GPU technology can further enhance their performance in terms of computation time.