The classic approach to image matching consists in the detection, description and matching of keypoints. This defines a zero-order approximation of the mapping between two images, determined by corresponding point coordinates. But the patches around keypoints typically contain more information, which may be exploited to obtain a first-order approximation of the mapping, incorporating local affine maps between corresponding keypoints. In this work, we propose a Local Affine Transform Estimator (LATE) method based on neural networks. We show that LATE drastically improves the accuracy of local geometry estimation between images when compared to the state of the art. The second contribution of this paper consists of two modifications to the RANSAC algorithm, that use LATE to improve the homography estimation between a pair of images. Our experiments show that these approaches outperform RANSAC for different choices of image descriptors and image datasets, and permit to increase the number of correctly matched image pairs in challenging matching databases.