Diffusion magnetic resonance imaging (dMRI) tractography is a powerful approach to study brain structural connectivity. However, its reliability in a clinical context is still highly debated. Recent studies have shown that most classical algorithms achieve to recover the majority of the existing true bundles. However, the generated tractograms contain many invalid bundles. This is due to the crossing fibers and bottleneck problems which increase the number of false positives fibers. In this work, we proposed to overpass this limitation with a novel method to guide the algorithms in those challenging regions with prior knowledge of the anatomy. We developed a method of creating and combination of anatomical prior applicable to any orientation distribution function (ODF)-based tractography algorithms. The proposed method captures the track orientation distribution (TOD) from an atlas of segmented fiber bundles and incorporates it during the tracking process, using a Riemannian framework. We tested the prior incorporation method on two ODF-based state-of-the-art algorithms, iFOD2 and Trekker PTT, on the diffusion-simulated connectivity (DiSCo) dataset and on the Human Connectome project (HCP) data. We showed that our method improves the overall spatial coverage and connectivity of a tractogram on the two datasets, especially in crossing fiber regions. Moreover, the fiber reconstruction may be improved on clinical data, informed by prior extracted on high quality data, and therefore could help in the study of brain anatomy and function.