We derive the predicted time dilation of delocalized atomic clocks in an optical lattice setup in the presence of a gravitational field to leading order in quantum relativistic corrections. We investigate exotic quantum states of motion whose gravitational time dilation is outside of the realm of classical general relativity, finding a regime where $^{24}\mathrm{Mg}$ optical lattice clocks currently in development would comfortably be able to detect this quantum effect (if the technical challenge of generating such states can be met). We provide a detailed experimental protocol and analyse the effects of noise on our predictions. We also show that the magnitude of our predicted quantum gravitational time dilation effect remains just out of detectable reach for the current generation of $^{87}\mathrm{Sr}$ optical lattice clocks. Our calculations agree with the predicted time dilation of classical general relativity when restricting to Gaussian states.