In this paper, we proposed a Markov Random field sequence segmentation and regions tracking model, which aims at combining color, texture, and motion features. First a motion-based segmentation is realized. The global motion of the video sequence is estimated and compensated. From the remaining motion information, the motion segmentation is achieved. Then, we use a Markovian approach to update and track over time the video objects. By video object, we mean typically, a spatio-temporal shape characterized by its texture, its color, and its motion. The temporal map is updated and propagated using our Markov Random Field segmentation model to keep consistency in video objects tracking.