Although considered as not safety critical (safety function being performed by other sub-systems), railway supervision systems can contribute to hazardous scenarios. It is of primary importance to identify this type of scenarios and evaluate the behaviour of human operators. A state of the art in human reliability is presented. This article presents an experimental protocol based on an automatic train supervision system coupled to a traffic simulator. It allows to gather data for human reliability evaluation and man-machine performance study.