The accuracy of forest classification is generally improved by multisensor data fusion since tree species identification benefits from complementary information. However, hypothesizing multisource fusion can also deteriorate accuracy when a non-relevant source is added, we propose a fusion method for classes in difficulty. When the difficulty threshold we introduce is appropriated, our method outperforms the classical approach consisting in performing fusion for all classes. Moreover, the fusion processing time can widely decrease when several classes are put aside. This method can be used effectively to enhance accuracy and processing speed when analyzing the wealth of information available from remote sensors.