With the constant evolution of CCTV cameras deployed in major cities to ensure the citizens' security, CCTV operators have to watch a huge amount of video when they are searching for scenes, objects, or target persons. Watching or processing some video sequences can be useless for several reasons: content is unsuitable for operators' needs, unusable shooting conditions, etc. Filtering useless content can be an efficient way for operators to save time. In this paper we propose an approach for CCTV contents filtering based on contextual information in order to provide CCTV operators with video sequences of interest. The proposed approach takes into account many sources of contextual information such as: open data, social media, mobility, geolocation, and crowdsourcing. We provide an analysis of contextual information relevant for this approach. Since interoperability is one of the main problems of context-based approaches, we propose a generic data model of contextual information used in our approach in order to tackle this issue. Based on this data, we propose a framework architecture for relevant video segments recommendation.