In the quest for thinner and more efficient ferroelectric devices, Hf0.5Zr0.5O2 (HZO) has emerged as a potential ultrathin and lead-free ferroelectric material. Indeed, when deposited on a TiN electrode, 1–25 nm thick HZO exhibits excellent ferroelectricity capability, allowing the prospective miniaturization of capacitors and transistor devices. To investigate the origin of ferroelectricity in HZO thin films, we conducted a far-infrared (FIR) spectroscopic study on 5 HZO films with thicknesses ranging from 10 to 52 nm, both within and out of the ferroelectric thickness range where ferroelectric properties are observed. Based on X-ray diffraction, these HZO films are estimated to contain various proportions of monoclinic (m-), tetragonal (t-), and polar orthorhombic (polar o-) phases, while only the 11, 17, and 21 nm thick are expected to include a higher amount of polar o-phase. We coupled the HZO infrared measurements with DFT simulations for these m-, t-, and polar o-crystallographic structures. The approach used was based on the supercell method, which combines all possible Hf/Zr mixed atomic sites in the solid solution. The excellent agreement between measured and simulated spectra allows assigning most bands and provides infrared signatures for the various HZO structures, including the polar orthorhombic form. Beyond pure assignment of bands, the DFT IR spectra averaging using a mix of different compositions (e.g., 70% polar o-phase +30% m-phase) of HZO DFT crystal phases allows quantification of the percentage of different structures inside the different HZO film thicknesses. Regarding the experimental data analysis, we used the spectroscopic data to perform a Kramers–Kronig constrained variational fit to extract the optical functions of the films using a Drude–Lorentz-based model. We found that the ferroelectric films could be described using a set of about 7 oscillators, which results in static dielectric constants in good agreement with theoretical values and previously reported ones for HfO2-doped ferroelectric films.