Safety is a key issue in railway operation. In this context, level crossings (LCs) are one of the most critical points in railway networks. In some countries, accidents at LC account for up to 50% of railway accidents. In this paper, we conduct a risk assessment comparative study involving two main types of Automatic Protection Systems (APSs), the first using a pair of half-barriers and the second with four half-barriers. So far, the choice of such LC protection systems has been exclusively done on the basis of qualitative expertise. The study we carry out here is based on some parameterizable behavioral models we have developed, which describe the global dynamics within the LC area. In contrast to existing studies on LC safety, our models take into account not only railway and road traffic but also the risk due to human factors while focusing on two major risky situations. The simulation results clearly show the potential risk with each of the investigated APSs, according to various features of the dynamics within the LC area. To the best of our knowledge, this is the first work dealing with a quantitative comparison between different types of LCs. The developed models can be easily accommodated in order to describe existing infrastructures.