Towards Semantic and Self-Supervised Learning Approach for Anomaly Detection on Video Streams
Résumé
Video anomaly detection has proved to be a challenging task over the last decades. One of the applications that raises great interest is the video surveillance field. However, the great potential of improving situation awareness, prediction, and prevention of undesirable events in various contexts is still far from fully explored. Thus, there is a significant need for approaches and methods able to understand the semantics behind a video and extract hidden patterns and knowledge. For that purpose, there has been a recent interest in using scene graphs for tasks involving computer vision as they are interpretable and can enhance semantics behind the scenes of the video. Using this intuition, we propose to tackle this issue by exploring a semantic-aware method for video anomaly detection that allows identifying entities involved in the scene presenting abnormal contexts. With the help of a scene graph and a self-supervised contrastive learning approach based on the Deep Graph Infomax (DGI) algorithm, we can encode the normal or the most common contexts behind the video and discriminate abnormal ones. Thus, entities or relationships that are semantically considered distinct from other entities are then considered as anomalous. The obtained detections and results are very satisfying compared to the state-of-the-art approaches with the capacity to enhance the understanding and the interpretability of the video scene. Keywords: Self-supervised and contrastive learning, semantic-aware, video anomaly detection, interpretability.