Convolutional Neural Networks (CNNs) are now commonly used in the computer vision community, in particular for optical flow estimation. Some attempts to use such tools to measure displacement and strain fields from pairs of reference/deformed speckle images (like Digital Image Correlation) have been recently reported in the literature. The aim of this work is twofold. The first one is to customize a state-of-the-art CNN dedicated to optical flow estimation to reach better performance when processing speckle images. This is mainly obtained by removing the deepest levels. The second one is to further simplify the CNN by reducing as much as possible the number of filters in the remaining levels while keeping equivalent metrological performance to the original version, in order to accelerate image processing on a power-efficient compact Graphics Processing Unit (GPU). Synthetic images deformed through a suitable displacement field are used to assess the metrological performance of the different versions of the CNN tested in this study. We focus on the sub-pixel part of the displacement is considered for this first attempt, this part being much more challenging to determine than integer displacements obtained at the pixel scale. The latter can be found by cross-correlation or with a rough version of DIC. Real images are tested with the simplest version of the CNN and obtained results are compared with those provided by classic subset-based Digital Image Correlation. The two main conclusions are i-that the customization procedure improves the metrological performance of the original version and ii-that the metrological performance of the ultimate simplified version of the CNN is globally equivalent to the one of the initial version despite the drastic simplification obtained at the end of the procedure. This performance lies between that of DIC used with first-and second-order subset shape functions.