The magnetodielectric studies on hexagonal Ho1−xDyxMnO3 reveal an overall enhancement of magnetoelectric coupling upon Dy3+ substitution. However, the magnitude of the enhancement is varied with doping level x. Magnetodielectric response is larger for x = 0.1 and it started decreasing with further increase in x. We suggest that the specific site (C3V site) substitution of higher ionic radii Dy3+ in the structure could play a key role in defining magnetoelectric coupling strength through structural distortion. The evolution of c/a ratio which is a measure of distortion in the hexagonal lattice agrees well with the observed percentage change in the dielectric constant which in turn substantiates the role of lattice distortion on the enhancement in magnetodielectric effect.