The problem of temporal data clustering is addressed using a dynamic Gaussian mixture model. In addition to the missing clusters used in the classical Gaussian mixture model, the proposed approach assumes that the means of the Gaussian densities are latent variables distributed according to random walks. The parameters of the proposed algorithm are estimated by the maximum likelihood approach. However, the EM algorithm cannot be applied directly due to the complex structure of the model, and some approximations are required. Using a variational approximation, an algorithm called VEM-DyMix is proposed to estimate the parameters of the proposed model. Using simulated data, the ability of the proposed approach to accurately estimate the parameters is demonstrated. VEM-DyMix outperforms, in terms of clustering and estimation accuracy, other state-of-the-art algorithms. The experiments performed on real world data from two fields of application (railway condition monitoring and object tracking from videos) show the strong potential of the proposed algorithms.