Identifying objects from a video stream is a fundamental and critical task in many computer-vision applications. A popular approach is the background subtraction, which consists in separating foreground (moving objects) from background. Many methodologies have been developed for automatic background segmentation but this fundamental task is still challenging. We focus here on a particular application of computer vision: intrusion detection in video surveillance. We propose in this paper a multi-level methodology for evaluating and comparing background subtraction algorithms. Three levels are studied: first, pixel level to evaluate the accuracy of the segmentation algorithm to attribute the right class to each pixel. Second, image level, measuring the rate of right decision on each frame (intrusion vs no intrusion) and finally sequence level, measuring the accordance with the time span where objects appear. Moreover, we also propose a new similarity measure, called D-Score, adapted to the context of intrusion detection.