Segmentation and tracking of tagged MR images is a critical component of in vivo understanding for the heart dynamics. In this paper, we propose a novel approach which uses multi-dimensional features and casts the left ventricle (LV) extraction problem as a maximum posteriori estimation process in both the feature and the shape spaces. Exact integration of multi-dimensional boundary and regional statistics is achieved through a global formulation. Prior is enforced through a point-distribution model, where distances between landmark positions are learned and enforced during the segmentation process. The use of divergence theorem leads to an elegant pairwise formulation where image support and prior knowledge are jointly encoded within a pairwise MRF and the segmentation is achieved efficiently by employing MRF inference algorithms. Promising results on numerous examples demonstrate the potentials of our method.