Low-dose CT-like imaging systems offer numerous perspectives in terms of clinical application, in particular for osteoarticular diseases. In this paper, we address the challenging problem of 3D femur modeling and estimation from bi-planar views. Our contributions are threefold. First, we propose a non-uniform hierarchical decomposition of the shape prior of increasing clinical-relevant precision which is achieved through curvature driven unsupervised clustering acting on the geodesic distances between vertices. Second, we introduce a graphical-model representation of the femur which can be learned from a small number of training examples and involves third-order and fourth-order priors, while being similarity and mirror-symmetry invariant and providing means of measuring regional and boundary supports in the bi-planar views. Last but not least, we adopt an efficient dual-decomposition optimization approach for efficient inference of the 3D femur configuration from bi-planar views. Promising results demonstrate the potential of our method.