We show that the ghost degrees of freedom of Einstein gravity with a Weyl term can be tamed by a simple mechanism that invokes local Lorentz symmetry breaking. We demonstrate how the mechanism works in a cosmological setting. The presence of the Weyl term forces a redefinition of the quantum vacuum state of the tensor perturbations. As a consequence the amplitude of their spectrum blows up when the Lorentz-violating scale becomes comparable to the Hubble radius. Such a behaviour is in sharp contrast to what happens in standard Weyl gravity where the gravitational ghosts smoothly damp out the spectrum of primordial gravitational waves.