Image registration represents one of the fundamental techniques in medical imaging and image-guided interventions. In this paper, we present a Convolutional Neural Network (CNN) framework for deformable transesophageal US/CT image registration, for the cardiac arrhythmias, and guidance therapy purposes. The framework consists of a CNN, a spatial transformer, and a resampler. The CNN expects concatenated pairs of moving and fixed images as its input, and estimates as output the parameters for the spatial transformer, which generates the displacement vector field that allows the resampler to wrap the moving image into the fixed image. In our method, we train the model to maximize standard image matching objective functions that are based on the image intensities. The network can be applied to perform non-rigid registration of a pair of CT/US images directly in one pass, avoiding so the time consuming computation of the classical iterative method.