We propose a new image analysis method to segment and track cells in a growing colony. By using an intermediate low-dimension image representation yielded by a reliable over-segmentation process, we combine the advantages of two-steps methods (possibility to check intermediate results) and the power of simultaneous segmentation and tracking algorithms, which are able to use temporal redundancy to resolve segmentation ambiguities. We improve and measure the tracking performances with a notion of decision risk derived from cell motion priors. Our algorithm permits to extract the complete lineage of a growing colony during up to seven generations without requiring user interaction.