The use of high-performance perovskite-structure epitaxial relaxor ferroelectric films [S. H. Baek et al., Science 334, 958 (2011)] is hindered by the lack of knowledge of epitaxial effects therein. It is experimentally shown here that a biaxial epitaxial compression can favor the relaxor state over ferroelectricity. The absence of the ferroelectric transition and the existence of the low-temperature relaxor state are evidenced by a combination of x-ray diffraction, dielectric, polarization, and optical studies of PbSc0.5Nb0.5O3 films epitaxially grown on La0.5Sr0.5CoO3/MgO(001). This finding is beyond existing models of polarization in perovskite-structure epitaxial films and beyond the established ability to induce ferroelectricity by an epitaxial strain.