Direct detection and spectroscopy of exoplanets requires high-contrast imaging. For habitable exoplanets in particular, located at a small angular separation from the host star, it is crucial to employ small inner working angle (IWA) coronagraphs that efficiently suppress starlight. These coronagraphs, in turn, require careful control of the wavefront that directly impacts their performance. For ground-based telescopes, atmospheric refraction is also an important factor, since it results in a smearing of the point-spread function (PSF), that can no longer be efficiently suppressed by the coronagraph. Traditionally, atmospheric refraction is compensated for by an atmospheric dispersion compensator (ADC). ADC control relies on an a pri ori model of the atmosphere whose parameters are solely based on the pointing of the telescope, which can result in imperfect compensation. For a high-contrast instrument like the Subaru Coronagraphic Extreme Adaptive Optics (SCExAO) system, which employs very small IWA coronagraphs, refraction-induced smearing of the PSF has to be less than 1 mas in the science band for optimum performance. In this paper, we present the first on-sky measurement and correction of residual atmospheric dispersion. Atmospheric dispersion is measured from the science image directly, using an adaptive grid of artificially introduced speckles as a diagnostic to feedback to the telescope’s ADC. With our current setup, we were able to reduce the initial residual atmospheric dispersion from 18.8 mas to 4.2 in broadband light (y- to Hband) and to 1.4 mas in the H-band only. This work is particularly relevant to the upcoming extremely large telescopes (ELTs) that will require fine control of their ADC to reach their full high-contrast imaging potential.