For a solar adaptive optics system equipped with a Shack-Hartmann wavefront sensor, the local wavefront slopes are measured from the displacements of the images produced by the sub-pupils of the sensor with respect to a given reference image. Measuring these displacements is challenging because of the very low contrast (at most a few percent) of the structures at the Sun surface. Additional difficulties arise from the fact that these structures evolve in time and are slightly distorted in each sub-image. In this contribution, we describe a novel approach to process the images of a solar wavefront sensor which jointly estimates the wavefront slopes, their noise covariance matrix, and the reference image. Spatio-temporal constraints are imposed on the reference image to regularize the problem and stabilize the global tip-tilt. Automatically tuned correction factors are introduced to account for the scintillation and the local distortions. Our method yields a sufficient statistic which enables an optimal wavefront reconstruction. We propose an alternating strategy to quickly solve the joint estimation problem. Special attention has been paid to make the numerical algorithm usable in real-time. Our method is implemented in the adaptive optics system of the THEMIS solar telescope equipped with a 10 × 10 Shack-Hartmann wavefront sensor delivering 400 × 400 pixel images at 1kHz. On a single CPU core, the Julia version of our algorithm provides the measurements with 90μs of latency after the image acquisition and takes an additional 200µs to update the reference image.