We address the issue of the origin of the phonon thermal Hall effect in Tb3Ga5O12, an intriguing property presumed to originate from magnetoelastic properties, and magnified in this compound by the non-Kramers nature of Tb3+ ions. Using neutron scattering, we have explored both the spin and lattice dynamics of Tb3Ga5O12. Our experimental results show that the transition toward the magnetic ground state, below TN = 280 mK, is driven by the softening of an exciton, as expected in a two-singlet system like Tb3Ga5O12. Low-energy excitations in the ordered phase are still excitons, whose dispersion throughout the Brillouin zone is driven by magnetic interactions. We have also discovered a mixing between specific phonon and exciton modes, this hybridization being evidenced through an intensity anomaly of the transverse acoustic phonons, as they cross low-energy crystal field excitations. Those experimental results can be comprehended by random phase approximation calculations, involving a Hamiltonian based on crystal electric field, dipolar interactions, and a coupling between phonons and the quadrupolar 4 f electronic density.