Punctate white matter lesions (PWML) are the most common white matter injuries observed in preterm neonates. Automatic detection of these lesions could better assist doctors in diagnosis. Recent advances in deep learning have resulted in optimistic results on many MR biomedical image benchmark datasets, but few methods seem to tackle the detection of very small lesions in ultrasound images. In this paper, we propose a two-phase strategy. Firstly, we highlight the foreground information by aggregating the lesions in the ground truth along the coronal projection of the brain, then we train a segmentation network to detect PWML with the resulting oversegmented masks. Secondly, we introduce a novel deep architecture for multimodal classification, called 2.5D SC-Net, which is used to eliminate false alarms and improve specificity. Experimental results demonstrate the effectiveness of our method to detect PWML in ultrasound images, improving the recall by 15% compared to the best published models, while limiting the number of false alarms efficiently.