Heat assisted magnetic recording (HAMR) is a new technology which uses temporary near field laser heating of the media during write to increase hard disk drive storage density. By using plasmonic antenna embedded in the write head, extremely high thermal gradient are created in the recording media (up to 10K/nm). State of the art HAMR media consists of grains of FePtX ordered alloys exhibiting high perpendicular anisotropy separated by insulating grain boundaries. Nearby the plasmonic antenna, the difference of temperature between two 8nm wide neighboring grains in the media can reach 80K, representing a gigantic thermal gradient of ~40K/nm across the grain boundary. Such situations with much weaker thermal gradient (~1K/nm, already considered as very large) have already been studied in the field of spincaloritronics. There, it was shown that very large spin transfer torques due to thermal gradients can arise in magnetic tunnel junctions which can even yield magnetization switching. Considering that two neighboring grains separated by an insulating grain boundary in a HAMR media can be viewed as a magnetic tunnel junction and that the thermal gradients in HAMR are one to two orders of magnitude larger than those existing in conventional spincaloritronics, one may expect a major impact from these thermal torques on magnetization switching dynamics and therefore on HAMR recording performances. This paper combines theory, experiments aiming at determining the polarization of tunneling electrons across the media grain boundaries, and micromagnetic simulations of recording process. It is shown that the thermal in-plane torque can have a detrimental impact on the recording performances by favoring antiparallel magnetic alignment between neighboring grains during the media cooling. Implications on media design are discussed in order to limit the impact of these thermal torques.