In this work we develop a new phase field method for modeling fracture in polycrystalline materials. Many studies addressed this problem by combining cohesive models at the grain boundaries and damage models for the bulk. In this paper, a unified formulation is proposed in the phase field framework. Anisotropic failure is considered within the grain. Each of the preferential failure direction is associated with a damage variable. An additional damage variable is dedicated to the grain boundaries which are considered of finite thickness. Furthermore the value of the fracture energy at the grain boundaries is set depending on the local misorientation. The obtained model allows modeling competition and interactions between intergranular and transgranular fracture. The proposed model is illustrated through several numerical examples including crack initiation and propagation in polycrystalline aggregates.