The aim of this study is to analyse the influence of both the microstructure and defects on the high cycle fatigue behaviour of the 316L austenitic stainless steel, using finite element simulations of polycrystalline aggregates. High cycle fatigue tests have been conducted on this steel under uniaxial (push-pull) and multiaxial (combined in-phase tension and torsion) loading conditions, with both smooth specimens and specimens containing artificial semi-spherical surface defects. 2D numerical models, using a cubic elastic constitutive model, are created to determine the degree of heterogeneity of the local stress parameters as a function of the defect size. This has been done for one microstructure using several orientation sets generated from the initial texture of the material. The grains are explicitly modelled and the anisotropic behaviour of each FCC crystal is described by the generalized Hooke’s law with a cubic elasticity tensor. From the simulations carried out with different defect sizes and orientation sets that are representative of the real texture of the tested material, statistical information regarding mesoscopic mechanical fields provides useful insight into the microstructural dependence of the driving forces for fatigue crack nucleation at the mesoscopic scale (or the scale of individual grains). The results in terms of the stress fields and fatigue crack initiation conditions are determined at both the mesoscopic and macroscopic scales. The results from these FE models are used along with an original probabilistic mesomechanics approach to quantify the defect size effect. The resulting predictions, which are sensitive to the microstructure, include the probability distribution of the high cycle fatigue strength.