A physical model simulating the spatio-temporal distribution of temperature in an ellipsoidal fruit has been developed. It is based on the numerical resolution of the three-dimensional (3D) unsteady heat conduction equation with unsteady and non-homogeneous heat fluxes as boundary conditions. The numerical scheme and the physical models have been tested by comparing with an analytical solution for simple configurations. The model quality has been assessed by comparing model outputs to temperature measured with thermocouples at several locations in isolated peach and apple fruits. The root mean square error (RMSE) for temperature simulated at a 1 min time step was about 0.8 °C. A sensitivity analysis showed that accurate estimations of the surface conductance to water vapor diffusion and of the thermal conductivity are necessary. The model has also been used for identifying the main microclimate variables governing temperature dynamics within fruits.