An anomalous event is commonly defined as an event sensibly distinct from the majority of its counterparts in a given context. Hence, video anomaly detection is often tackled as an out-of-distribution problem. Recent self-supervised state-of-the-art anomaly detection approaches are trained on object-centric descriptors extracted using object detectors. While these approaches are efficient for simultaneous detection and localization of single instance-related anomalies, they are not suited to identify anomalies related to different instance categories. In addition, anomalies that emerge from multiple instances interaction remain an open issue. In particular, we investigate the detection of anomalous interactions between pedestrians and automatic doors in the context of train video-surveillance. We propose a three parts approach. A panoptic segmentation network extracts instanceaware semantic maps of pedestrians and automatic doors in the input video sequence. Two self-supervised multi-tasks networks are trained separately on each semantic maps sequence using a set of proxy tasks specifically tailored for the considered object categories. Finally, both networks anomalous binary responses are fused to provide a final interaction anomaly detection classification. We evaluate our method on a railway application dataset to detect doors-pedestrian anomalous interactions.