FeRh is well known in its bulk form for a temperature-driven antiferromagnetic (AF) to ferromagnetic (FM) transition near room temperature. It has aroused renewed interest in thin film form, with particular focus on its biaxial AF magnetic anisotropy which could serve for data encoding, and the possibility to investigate laser assisted phase transitions, with varying contributions from electrons, phonons and magnons. In order to estimate the typical temperature increase occurring in these experiments, we performed modulated thermoreflectance microscopy to determine the thermal conductivity κ of FeRh. As often occurs upon alloying, and despite the good crystallinity of the layer, κ was found to be lower than the thermal conductivities of its constituting elements. More unexpectedly given the electrically more conducting nature of the FM phase, it turned out to be three times lower in the FM phase compared to the AF phase. This trend was confirmed by examining the temporal decay of incoherent phonons generated by a pulsed laser in both phases. To elucidate these results, first and second principles simulations were performed to estimate the phonon, magnon and electron contributions to the thermal conductivity. They were found to be of the same order of magnitude, and to give a quantitative rendering of the experimentally observed κAF. In the FM phase however, simulations overestimate the low experimental values, implying very different (shorter) electron and magnon lifetime