A novel mid-level video indexing method based on detection and tracking human faces is presented. Instead of detecting the faces on every frame, our method first detects the faces and then tracks them. Compared to our previous general-purpose tracking method, our approach is improved by: i) a Multi-Object model extension to track several objects in parallel; ii) a Dual Consistency Check by Kolmogrov-Smirnov test to alarm a scene change so as to stop the tracking and wait until the next detection; ii) application of temporal median filtering of initial detection by Viola $\&$ Jones detector. The combination of filtered detection and our tracking method evaluated on an excerpt of TRECVID 2009 database increases the F-measure by 7% compared to Viola & Jones detector alone.