We present a deep learning method to downscale low-resolution geophysical fields by merging them with high-resolution data. The downscaling was performed using an ensemble of convolutional neural networks (CNNs), whose prediction values are the average values of the outputs of 20 CNNs. Academic experiments were conducted on simulated ocean data in the Gulf Stream region, given by the outputs of the NATL60 model. The CNNs forced with low-resolution (120 × 120 km) sea surface high (SSH) data and mesoscale resolution (12 × 12 km) sea surface temperature (SST) data allowed us to obtain mesoscale resolution sea surface currents with good accuracy. Sensitivity experiments have shown that taking SST into account significantly increases the accuracy of the high-resolution velocity retrieval, even when noise is added to the SSH data. The velocity information embedded in the transport equation modeling the SST advection is taken into account by the CNN, which greatly increases the resolution of ocean currents provided by SSH. In the present work, we only consider spatial downscaling by assuming that SSH and SST are daily observations. The method we developed is generic and can be used to improve the resolution of a wide variety of large-scale fields by merging them with high-resolution fields.