High intensity transient signals (HITS) can be detected using transcranial Doppler ultrasound monitoring to help stroke prevention. The various types of HITS are by nature imbalanced: solid emboli are rare events, compared to artifacts or gaseous emboli. Therefore, when training deep models on these data, one have to take into account the class imbalance. In this work, we propose a deep embedded clustering (DEC) based regularization technique, DEC-R to handle imbalanced datasets. Our proposed method decompose a classification model into an encoder and a classifier, with DEC-R applied to the former during supervised training. We validate our method on four synthetic datasets and one HITS datasets. The results show that our approach is robust against imbalanced datasets, allowing to increase the classification performances of different models on both, imbalanced (maximum imbalance ratio of 20) and balanced datasets. We achieve an increase of 4.86% and 1.27% in terms of Matthews correlation coefficient on two imbalanced datasets, a synthetic dataset (of 3D points) and a HITS clinical dataset, respectively. Our code can be found in GitHub.