In this work, we propose a new approach to recover trajectories from points previously detected in a sequence of images. In presence of spurious and missing detections, actual trajectories can be characterized by an a-contrario model, that estimates the probability of observing a similar trajectory in random data. This results in a single criterion combining trajectory characteristics (duration, number of points, smoothness) and data statistics (number of images and detected points), which can then be used to drive a dynamic programming algorithm able to extract sequentially the most meaningful trajectories. The performances obtained on synthetic and real-world data are studied in detail, and shown to compare favorably to the state-of-the-art ROADS algorithm.