The structure of lithium tantalate powders has been investigated using neutron diffraction between room temperature and 445 K, which includes the zero-birefringence point. As the temperature increases, the displacement of the Ta atom from the centre of the O octahedra and the tilt of the octahedra both decrease linearly in the range investigated. The measured structures form the basis of a range of density functional theory calculations utilizing the WIEN2 k code, with a focus on calculating the optical properties. These calculations are sensitive to the small structural changes measured in this temperature range; the calculated birefringence changes are consistent with the changes measured experimentally. Further, by investigating the effect of each atom in isolation, it can be shown that the birefringence of lithium tantalate mainly depends on the Ta displacement and the octahedral tilt, with the linear change in these as a function of temperature producing the change in birefringence with temperature, which results in it becoming zero-birefringent. This work demonstrates the unique material insights that can be obtained by combining density functional calculations with precise structural studies.