We study the angular dynamics of small non-spherical particles settling in a turbulent flow, such as ice crystals in clouds, aggregates of organic material in the oceans, or fibres settling in turbulent pipe flow. Most solid particles encountered in Nature are not spherical, and their orientations affect their settling speeds, as well as their collision and aggregation rates in suspensions. Whereas the random action of turbulent eddies favours an isotropic distribution of orientations, gravitational settling breaks the rotational symmetry. The precise nature of the symmetry breaking, however, is subtle. We demonstrate here that the fluid-inertia torque plays a dominant role in the problem. As a consequence rod-like particles tend to settle in turbulence with horizontal orientation, the more so the larger the settling number Sv (a dimensionless measure of the settling speed). For large Sv we determine the fluctuations around this preferential horizontal orientation for prolate particles with arbitrary aspect ratios, assuming small Stokes number St (a dimensionless measure of particle inertia). Our theory is based on a statistical model representing the turbulent velocity fluctuations by Gaussian random functions. This overdamped theory predicts that the orientation distribution is very narrow at large Sv, with a variance proportional to-Sv 4. By considering the role of particle inertia, we analyse the limitations of the overdamped theory, and determine its range of applicability. Our predictions are in excellent agreement with numerical simulations of simplified models of turbulent flows. Finally we contrast our results with those of an alternative theory predicting that the orientation variance is proportional to-Sv 2 at large Sv.