We prove the large deviation principle for several entropy and cross entropy estimators based on return times and waiting times on shift spaces over finite alphabets. In the case of standard return times, we obtain a nonconvex large-deviation rate function. We consider shift-invariant probability measures satisfying some decoupling conditions which imply no form of mixing nor ergodicity. We establish precise relations between the rate functions of the different estimators, and between these rate functions and the corresponding pressures, one of which is the Rényi entropy function. The results apply in particular to irreducible Markov chains, equilibrium measures for Bowen-regular potentials, g-measures, invariant Gibbs states for summable interactions in statistical mechanics, and also to probability measures that may be far from Gibbsian, including some hidden Markov models and repeated quantum measurement processes.