This paper aims to develop a suite of customizable Petri net benchmarks to deal with various issues pertaining to fault diagnosis in discrete event systems. As reported in previous studies, the existing benchmarks generally characterize some of the numerous issues concerning fault diagnosis. This satisfies a specific case study, whereas it may constitute a barrier when performing comparative analysis between different approaches. In order to bridge this gap, we propose to develop a suite of benchmarks based on the same case study, while covering a wide spectrum of diagnosis issues, namely diagnosability according to the original definition, K-diagnosability, diagnosability under fairness conditions, of intermittent faults and in timed context. The benchmark elaborated, based on a level crossing case study, can be enlarged to deal with the combinatorial explosion problem. In addition, it can be extended to integrate additional features. A software tool is provided to generate benchmarks, according to the target issues one seeks to tackle, and the desired size of the model.