We study the crushing strength of brittle materials whose internal structure (e.g., mineral particles or grains) presents a layered arrangement reminiscent of sedimentary and metamorphic rocks. Taking a discrete-element approach, we probe the failure strength of circular-shaped samples intended to reproduce specific mineral configurations. To do so, assemblies of cells, products of a modified Voronoi tessellation, are joined in mechanically-stable layerings using a bonding law. The cells’ shape distribution allows us to set a level of inherent anisotropy to the material. Using a diametral point loading, and systematically changing the loading orientation with respect to the cells’ configuration, we characterize the failure strength of increasingly anisotropic structures. This approach lets us reproduce experimental observations regarding the shape of the failure strength curve, the Weibull modulus, failure patterns of rocks, and quantify the consumption of the fragmentation energy, and the induced anisotropies linked to the cell’s geometry and force transmission in the samples. Based on a fine description of geometrical and mechanical properties at the onset of failure, we develop a micromechanical breakdown of the crushing strength variability using an analytical decomposition of the stress tensor and the geometrical and force anisotropies. We can conclude that the origins of failure strength in anisotropic layered media rely on compensations of geometrical and mechanical anisotropies, as well as an increasing average radial force between minerals indistinctive of tensile or compressive components.