Numerous power-based wind turbine (WT) fault-detection methods using supervisory control and data acquisition (SCADA) data are presented in the literature. However, their performance cannot be compared easily with one another because of the lack of a realistic benchmark. To address this concern, a novel and realistic simulation framework is presented. It utilises actual data recorded on five French wind farms located at different geographical sites and composed of WTs of different models. It was used to simulate power profiles for three-year data, generated from 25 different wind and temperature profiles on 25 different WTs. Thus, the benchmark enabled a rigorous comparison of the performances of power-based fault-detection solutions. The fault-detection performances of three detection methods were compared for four power-based fault and under-performance scenarios of various intensities. The results indicated that the fault-detection performance of a method can vary considerably depending on the environmental and operational conditions. Moreover, the most effective approach is the one that considers these operational and environmental variations in WT data. The detection performance for the four failure scenarios was also statistically analysed.