A major challenge in the macroscopic modeling of brittle failure initiation is to reconcile stress-driven failure in absence of stress concentration and energy-driven failure under high stress concentration (crack). In this paper, we perform athermal molecular simulations to investigate the underlying physics behind stress- to energy-driven failures. In the athermal limit, the evolution of an atomic system is deterministic and is obtained by energy minimization. Failure is expected when the system suddenly bifurcates to a broken configuration which can be formally evaluated as an atomic instability characterized by a negative eigenvalue of the Hessian matrix. We applied this methodology to a 2D toy model and to pristine graphene. Both stress- and energy-driven failures are triggered by an instability at the atomic scale, but the two types of failure differ widely regarding the mechanisms of instability (eigenvectors) and their multiplicity (degeneracy). With respect to existing macroscopic theories of failure initiation, these results raise some issues. In particular, one should distinguish the initiation mechanisms and the physical cracking occurring after initiation, and the spatial extent of the initiation mechanism should depend on stress concentration with a minimum extent given by the ratio between toughness and strength. From an atomic scale perspective, a strain-based stability formulation seems the most appropriate. Finally, we show that the degeneracy of the modes of failure explains the size-scaling of strength and toughness at finite temperature.