The objective of the system described here is to develop a warning system based on intelligent detection of potentially dangerous situations occurring at Level crossings (LC) and some hazard situations in the larger surrounding of the LC. It identifies and detects in real time potential and principle factors of accidents at LC and shares such dangerous situations to alert as LC safety measures. A global architecture of the video detection system is defined. Algorithms are developed for detection of potentially dangerous situations. Datasets of different dangerous situations are collected to validate the proposed system. The system is developed in the framework of the H2020 European Project called SAFER-LC.