Saturn's axial tilt of 26.7° produces seasons in a similar way as on Earth. Both the stratospheric temperature and composition are affected by this latitudinally varying insolation along Saturn's orbital path. The atmospheric thermal structure is controlled and regulated by the amount of hydrocarbons in the stratosphere, which act as absorbers and coolants from the UV to the far-IR spectral range, and this structure has an influence on the amount of hydrocarbons. We study here the feedback between the chemical composition and the thermal structure by coupling a latitudinal and seasonal photochemical model with a radiative seasonal model. Our results show that the seasonal temperature peak in the higher stratosphere, associated with the seasonal increase of insolation, is shifted earlier than the maximum insolation peak. This shift is increased with increasing latitudes and is caused by the low amount of stratospheric coolants in the spring season. At 80° in both hemispheres, the temperature peak at 10-2 mbar is seen to occur half a season (3-4 Earth years) earlier than was previously predicted by radiative seasonal models that assumed spatially and temporally uniform distribution of coolants. This shift progressively decreases with increasing pressure, up to around the 0.5 mbar pressure level where it vanishes. On the opposite, the thermal field has a small feedback on the abundance distributions. Accounting for that feedback modifies the predicted equator-to-pole temperature gradient. The meridional gradients of temperature at the mbar pressure levels are better reproduced when this feedback is accounted for. At lower pressure levels, Saturn's stratospheric thermal structure seems to depart from pure radiative seasonal equilibrium as previously suggested by Guerlet et al. (2014). Although the agreement with the absolute value of the stratospheric temperature observed by Cassini is moderate, it is a mandatory step toward a fully coupled GCM-photochemical model.