This work presents a new method for tracking and segmenting along time interacting objects within an image sequence. One major contribution of the paper is the formalization of the notion of visible and occluded parts. For each object, we aim at tracking these two parts. Assuming that the velocity of each object is driven by a dynamical law, predictions can be used to guide the successive estimations. Separating these predicted areas into good and bad parts with respect to the final segmentation and representing the objects with their visible and occluded parts permits handling partial and complete occlusions. To achieve this tracking, a label is assigned to each object and an energy function representing the multi-label problem is minimized via a graph cuts optimization. This energy contains terms based on image intensities, that enable segmenting and regularizing the visible parts of the objects. It also includes terms dedicated to the management of the occluded and disappearing areas, that are defined on the areas of prediction of the objects. The results on several challenging sequences prove the strength of the proposed approach.