This paper summarizes our contribution to the MSSEG-II MICCAI 2021 challenge. The aim is to segment new multiple sclerosis (MS) lesions using pairs of FLAIR MR images. Our approach is based on a 3D U-Net applied patch-wise to the images. In order to take into account both time-points, we simply concatenate the images along the channel axis before passing them to the 3D U-Net. The strong imbalance between positive and negative voxels, exhibited by the challenge data, makes training deep learning model di cult. Instead of using handcrafted priors like brain masks or multi-stage methods, we experiment with a novel modification to online hard example mining (OHEM), where we use an exponential moving average (i.e., its weights are updated with momentum) of our 3D U-Net to mine hard examples. Using a moving average instead of the raw model should allow smoothing its predictions and allowing it to give more consistent feedback for OHEM.