Learning from imbalanced datasets remains a significant challenge for real-world applications. The decoupled training approach achieves better performance among existing approaches for long-tail recognition. Moreover, there are simple and effective tricks that can be used to further improve the performance of decoupled learning and help models trained on long-tailed datasets to be more robust to the class imbalance problem. However, if used inappropriately, these tricks can result in lower than expected recognition accuracy. Unfortunately, there is a lack of comprehensive empirical studies that provide guidelines on how to combine these tricks appropriately. In this paper, we explore existing long-tail visual recognition tricks and perform extensive experiments to provide a detailed analysis of the impact of each trick and to propose an effective combination of these tricks for decoupled training. Furthermore, we introduce a new loss function called hard mining loss (HML), which is more suitable to learn the model to better discriminate head and tail classes. In addition, unlike previous work, we introduce a new learning scheme for decoupled training following an end-to-end process. We conducted our evaluation experiments on the CIFAR10, CIFAR100 and iNaturalist 2018 datasets. The results show that our method outperforms existing methods that address class imbalance issue for image classification tasks. We believe that our approach will serve as a solid foundation for improving class imbalance problems in other computer vision tasks.