This paper presents a deep-learning-based algorithm dedicated to the processing of speckle noise in phase measurements in digital holographic interferometry. The deep learning architecture is trained with phase fringe patterns including faithful speckle noise, having non-Gaussian statistics and non-stationary property, and exhibiting spatial correlation length. The performances of the speckle de-noiser are estimated with metrics, and the proposed approach exhibits state-of-the-art results. In order to train the network to de-noise phase fringe patterns, a database is constituted with a set of noise-free and speckled phase data. The algorithm is applied to de-noising experimental data from wide-field digital holographic vibrometry. Comparison with the state-of-the-art algorithm confirms the achieved performance.